TTIC 103 (CMSC 35420): Statistical Methods for Artificial Intelligence, Autumn 2007

Instructor: David McAllester

Textbook: Pattern Recognition and Machine Learning by Chris Bishop, Springer 2006

MWF 11:30-12:20 TTIC 230 (Press Buidling)

Course Description

Grading: The course will have roughly one homework set per week, a midterm, and a final.

Description: This course gives a survey of mathematical methods in statistical modeling, inference, and learning with an emphasis on techniques widely used in speech recognition, computational linguistics, computer vision, and computational biology. This course is aimed at providing students with a core understanding of statistical AI.

The following is subject to revision as the quarter progresses.

Midterm Review List

Topics:

· Convexity and Jensen's Inequality (Bishop equation 1.114)

· Information Theory (Bishop 1.6)

· Covariance and the Central Limit Theorem (Bishop 2.3 pages 78 to 84)

· Principle Component Analysis (PCA) (Bishop 12.1), Singular Value Decomposition (SVD) (not in Bishop), and Kernel PCA (Bishop 12.3)

· Linear Regression (Bishop 3.1 through 3.1.3 but not 3.1.4)

· Linear Classification (no analogous treatment in Bishop)

· Bias-Variance (Bishop 3.2)

· Generalization Bounds (no analogous treatment in Bishop)

· Kernel Methods (Bishop 6.1 and 6.2)

· Boosting (Bishop 14.3)

· Neural Networks and Gradient Descent (Bishop chapter 5)

· Hidden Markov Models. Viterbi and Forward-Backward (Bishop chapter 13 up to 13.3)

· Probabilistic Context Free Grammars (not in Bishop)

· K Means and Expectation Maximization (EM) (Bishop 9.1, 9.2 and 9.4)

· Graphical Models and Junction Trees (Bishop chapter 8)

· Loopy Belief Propagation (Bishop 8.4.7)

· Graph Cuts for Graphical Model Inference (not in Bishop)

· Structured Labels (not in Bishop)

· Linear Dynamical Systems and the Kalman Filter.

· Viterbi vs. A*

· Feature Selection