Big Data and Education


A Massive Online Open Textbook (MOOT)
Version 1.01
by Ryan Baker
in cooperation between Teachers College, Columbia University and the Columbia Center for New Media Teaching and Learning
with commentary by Luc Paquette and Ryan Baker (forthcoming)

Chapter 1: Prediction Modeling
Video 1: Introduction [YouTube] [pptx]
Video 2: Regressors [YouTube] [pptx]
Video 3: Classifiers part 1 [YouTube] [pptx]
Video 4: Classification in RapidMiner [YouTube] [pptx] [alert: this lecture has known flaws]
Video 5: Classifiers part 3 [YouTube] [pptx]
Video 6: Case study in classification [YouTube] [pptx]
Week One Commentary [pdf]

Chapter 2: Model Goodness and Validation
Video 1: Detector confidence [YouTube] [pptx]
Video 2: Diagnostic metrics: kappa and accuracy [YouTube] [pptx]
Video 3: Diagnostic metrics: ROC and A' [YouTube] [pptx]
Video 4: Diagnostic metrics: Correlation and RMSE [YouTube] [pptx]
Video 5: Cross-validation and over-fitting [YouTube] [pptx]
Video 6: Other validity considerations [YouTube] [pptx]

Chapter 3: Behavior Detecton
Video 1: Ground truth [YouTube] [pptx]
Video 2: Data synchronization [YouTube] [pptx]
Video 3: Feature engineering [YouTube] [pptx]
Video 4: Automated feature generation and selection [YouTube] [pptx]
Video 5: Knowledge engineering and data mining [YouTube] [pptx]

Chapter 4: Knowledge Inference
Video 1: Knowledge Inference [YouTube] [pptx]
Video 2: Bayesian Knowledge Tracing [YouTube] [pptx]
Video 3: Performance Factors Analysis [YouTube] [pptx]
Video 4: Item Response Theory [YouTube] [pptx]
Video 5: Advanced Bayesian Knowledge Tracing [YouTube] [pptx]

Chapter 5: Relationship Mining
Video 1: Correlation Mining [YouTube] [pptx]
Video 2: Causal Mining [YouTube] [pptx]
Video 3: Association Rule Mining [YouTube] [pptx]
Video 4: Sequential Pattern Mining [YouTube] [pptx]
Video 5: Network Analysis [YouTube] [pptx]

Chapter 6: Visualization
Video 1: Introduction to Educational Visualization and Learning Curves [YouTube] [pptx]
Video 2: Moment-by-Moment Learning Graphs [YouTube] [pptx]
Video 3: Scatter Plots, Heat Maps, and Parameter Space Maps [YouTube] [pptx]
Video 4: State Space Networks [YouTube] [pptx]
Video 5: Other Visualizations [YouTube] [pptx]

Chapter 7: Structure Discovery
Video 1: Clustering [YouTube] [pptx]
Video 2: Cluster Validation [YouTube] [pptx]
Video 3: Advanced Clustering Algorithms [YouTube] [pptx]
Video 4: Applications of Clustering in EDM [YouTube] [pptx]
Video 5: Factor Analysis [YouTube] [pptx]
Video 6: Knowledge Structure: Q-Matrixes [YouTube] [pptx]
Video 7: Knowledge Structures: Other Approaches [YouTube] [pptx]

Chapter 8: Advanced Topics
Video 1: Discovery with Models [YouTube] [pptx]
Video 2: Discovery with Models Case Study [YouTube] [pptx]
Video 3: Text Mining [YouTube] [pptx]
Video 4: Hidden Markov Models [YouTube] [pptx]
Video 5: Conclusions and Future Directions [YouTube] [pptx]

Upcoming in version 1.02: Additional commentaries by Luc Paquette and Ryan Baker.

Upcoming in version 1.03: Corrections to slides recommended by students.

Upcoming in future versions: Assignments, slides for use in lectures, forums, class cohorts, licensing, and more!

Acknowledgements: Sincerest thanks to Elle Wang, Michael Cennamo, Stephanie Ogden, Michael de Leon, Therese Condit, students who have recommended additions or corrections, and others.

An earlier version of these materials were offered through the Coursera platform in 2013 as "Big Data and Education".

Bugs? Errors? Email Ryan Baker.

Please cite this MOOT as Baker, R.S. (2014) Big Data and Education. New York, NY: Teachers College, Columbia University.

All materials here copyright Teachers College, Columbia University, and Columbia University, 2013-2014.