Course Schedule & Readings

EE511 -- Introduction to Statistical Learning -- Spring 2008

4/1: class overview, issues in statistical learning   (Duh)
Readings: HTF 1,2, DHS 1 [notes]

4/4: issues continued, classification with likelihood functions   (Ostendorf)
Readings: HTF 2, DHS 2 (excluding * sections) [notes]

4/8: issues continued, learning parametric models via MLE  (Duh)
Readings: HTF 8.2, DHS 3.1-3.2 [notes]

4/10: Sufficient statistics, EM Algorithm (Ostendorf)
Readings: HTF 8.5, DHS 3.6 [notes (corrected 4/21), DHS Table, More on EM]

4/15: EM Mixture example, Bayesian alternatives to MLE for training of parametric distributions (Duh)
Readings: HTF 8.3 [notes]

4/17: classification by linear functions (Ostendorf)
Readings: HTF 4 [notes]

4/22: online learning, linear regression
Readings: HTF 3 [notes]

• Online learning, perceptron algorithm
• Least squares, Ridge regression and shrinkage

4/24: Non-parametric Classification and Estimation
Readings: HTF 13; DHS 4.5 [notes]

• K-nearest-neighbor, kernels
• Theoretical results and practical considerations

4/29: Feature reduction & selection 
Readings: HTF 14.5 [notes]

• General principles for feature extraction
• Feature selection: filter/wrapper
• Decorrelating transformations: PCA, LDA

5/1: Performance Estimation  
Readings: HTF 7 [notes]

5/6: Model Comparison, Model Selection  
Readings: HTF 7

• Comparing classifiers [notes]
• Model bias and variance (revisited)
• Cross-validation
• AIC, BIC, MDL
[model selection notes]

5/8: Categorical variables, decision trees  
Readings: HTF9.2, DHS 8.1-8.4 [notes]

5/13: SVMs
Readings: HTF 12 [notes]

5/15: More on SVMs, Exam review
Readings: HTF 12 (cont.) [notes]

5/20: EXAM

5/22: Model combination
Readings: HTF 8.7, 10.1-10.6, DHS 9.5 [notes]

5/27: Artificial Neural Networks
Readings: HTF 4.5, 11; DHS 6 [notes]

5/29: Unsupervised Learning (clustering) for Continuous Inputs
Readings: HTF 14; DHS 10 [notes]

6/3: Semi-supervised learning
notes

6/5: Semi-supervised learning, wrap up
[notes] Related readings:

• A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in Proc. Conference on Computational Learning Theory, 1998.
• K. Nigam and R. Ghani, "Analyzing the effectiveness and applicability of co-training," in Proc. International Conference on Information and Knowledge Management, 2000.
• U. Brefeld and T. Scheffer, "Co-EM support vector learning," in Proc. International Conference on Machine Learning, 2004.

6/10: Project Presentations (3:30-6:20) (All Students)

Back to the EE511 Course Information Page