20134311: Introduction to Machine Learning
Winter 2013
Instructor: Greg Shakhnarovich, greg@ttic.edu
Monday, 14001600, Ziskind, Room 1 note change in location
Tuesday, 09001100, Ziskind, Room 1
First lecture: Monday December 31, 2012
Please fill out this online form as soon as possible
Scope
This courses covers supervised learning, primarily classification and regreession. Some topics in unsupervised learning that are central to supervised tasks, such as density estimation, are covered as well. The emphasis is on the fundamental principles in design of statistical methods, such as biasvariance tradeoff and overfitting, and on computational issues in applying these methods to real data sets. In the process of eploring these fundamental topics the course covers in detail modern machine learning tools: generative and discriminative regression and classification models, support vector machines, decision trees, neural networks, ensemble methods etc.
There are no formal prerequisites, but working knowledge of linear algebra and basic probability are necessary.
Books
There is no official textbook for the class, however some books could
be very helpful:
 C. M. Bishop Pattern Recognition
and Machine Learning 
 T. Hastie, R. Tibshirani and
J. H. Friedman
The Elements of Statistical Learning (available online)

 K. P. Murphy Machine Learning: a Probabilistic Perspective
a new book

The Matrix Cookbook
Course calendar
Lecture slides, handouts, homework assignments, etc.
Note: All handouts and lecture slides are in PDF format. Any
scheduling information posted for future dates should be treated as
tentative.
 Monday Dec 31
 Tuesday Jan 1
 Maximum likelihood and logloss
 Biasvariance tradeoff in estimation
 Overfitting and model complexity
 Lecture material: slides (with animations)
slides in handout format (no animations)
 Notes on bias/variance decomposition in regression
 Monday Jan 7
 Regularization; ridge regression and lasso
 Introduction to classification
 Lecture material: slides (with animations)
slides in handout format (no animations)
 Monday Jan 7
 Tuesday Jan 8
 Monday Jan 14
 Monday Jan 14
 Saturday Jan 19
 Monday Jan 21
 Sunday Jan 27, 9:0011:00, Feinberg Room B
 Monday Jan 28
 Monday Jan 28
 Tuesday Jan 29
 Advanced models: mixtures of experts, neural networks, structured prediction
 Lecture material: slides (with animations)
slides in handout format (no animations)
 Tuesday Feb 19, 10:0013:00, Feinberg Room B