Commerce and research are being transformed by data-driven discovery and prediction. Skills required for data analytics at massive levels – scalable data management on and off the cloud, parallel algorithms, statistical modeling, and proficiency with a complex ecosystem of tools and platforms – span a variety of disciplines and are not easy to obtain through conventional curricula. Tour the basic techniques of data science, including both SQL and NoSQL solutions for massive data management (e.g., MapReduce and contemporaries), algorithms for data mining (e.g., clustering and association rule mining), and basic statistical modeling (e.g., linear and non-linear regression).

Part 0: Introduction 

• Examples, data science articulated, history and context, technology landscape

Part 1: Data Manipulation at Scale

• Databases and the relational algebra  
• Parallel databases, parallel query processing, in-database analytics 
• MapReduce, Hadoop, relationship to databases, algorithms, extensions, languages  
• Key-value stores and NoSQL; tradeoffs of SQL and NoSQL

Part 2: Analytics 

• Topics in statistical modeling: basic concepts, experiment design, pitfalls
• Topics in machine learning: supervised learning (rules, trees, forests, nearest neighbor, regression), optimization (gradient descent and variants), unsupervised learning

Part 3: Communicating Results 

• Visualization, data products, visual data analytics  
• Provenance, privacy, ethics, governance 

Part 4: Special Topics

• Graph Analytics: structure, traversals, analytics, PageRank, community detection, recursive queries, semantic web
• Guest Lectures

We expect you to have intermediate programming experience and familiarity with databases, roughly equivalent to two college courses.  We will have four programming assignments: two in Python, one in SQL, and one in R. The target audience is undergraduate students across disciplines who wish to build proficiency working with large datasets and a range of tools to perform predictive analytics.

After taking this course, you may be interested in participating in the three-course Certificate in Data Science offered through the University of Washington Professional and Continuing Education program.  This online course will provide an overview and introduction to the more extensive material covered in that program, which offers classroom-based instruction by data scientists from Microsoft and other Seattle players, networking opportunities with peers, case studies from the “front lines,” and deep dives into selected topics.

There will be selected readings each week.  

We recommend, but do not require, that students refer to the book Mining of Massive Datasets by Anand Rajaraman and Jeff Ullman

The class will consist of lecture videos about 8 to 10 minutes in length. These will contain 1-2 integrated quizzes per video. Some of these videos will be given by guest lecturers from the data science community. 

There will be no formal exams or standalone quizzes. 

There will be eight total assignments of which two are optional. 

We will provide a virtual machine equipped with all necessary software, but you are permitted (and encouraged) to install software in your own environment as well. 

There will be four structured programming assignments: two in Python, one in SQL, and one in R. 

There will also be two open-ended assignments graded by peer assessment: one in visualization, and one in which you will participate in a Kaggle competition. 

Finally, there will be two optional assignments: One involving an open-ended real-world project submitted by external organizations with real needs, and one involving processing a large dataset on AWS.

Will I get a Statement of Accomplishment after completing this class?  

Yes. Students who successfully complete the class will receive a Statement of Accomplishment signed by the instructor.  

What resources will I need for this class? 

For this course, you will need an Internet connection and either a) the ability to run a virtual machine locally or b) the ability and knowledge to install the appropriate software yourself.  The software will include Python 2.7 (including various libraries), R, SQLite (or another database you are comfortable using).  You will also have the opportunity to install and work with Hadoop, but for logistics reasons, we will not require its use in an assignment.  Some assignments will be open-ended.

What level of programming experience should I have? 

We expect intermediate programming experience in some language and some familiarity with database concepts.  There will be programming assignments, but these are not designed to test knowledge of the language itself and will not involve using any esoteric features.  The languages we will use are Python, R, and SQL.