This course provides an introduction to graphical models, exact and approximate inference and learning algorithms and applications. We will cover Bayesian belief networks and Markov Random Fields and focus on recent advances in learning with graphical models, such as loopy belief propagation, variational approximations, Conditional Random Fields, large margin methods and kernel methods for graphical models. Topics will be motivated by applications in computation linguistics, speech, bioinformations and computer vision.
The course is mainly geared towards graduate students, but may also be suitable for advanced undergraduate students with a solid mathematical background. There are no strictly enforced prerequisites, but familiarity with probability theory, calculus, and linear algebra is a plus.
There is no textbook for this class. Reading material (tutorials, book chapters, journal/conference papers) will be provided either electronically or as hard-copies.
Organization
- Standard lecture format
- Additional discussion sessions
- Final student presentations
Grading
Participation: mandatory
Presentation:
40% Final Projects: 60%
Jan 3
Admistrivia and Course Overview
Directed and Undirected graphical modelsTutorial on GM [Murphy01]
slides
Generative Approaches
Jan 17
No class - BIRS Workshop 07w5078 Mathematical
Programming in Data Mining and Machine Learning
Jan 22
Exact Inference
Tutorial on Junction Tree Algorithm [Barber03]
Chap3 of Graphical Models [Lauritzen96]
(copies available at my office)slides Jan 26
Parameter Learning
Tutorial on EM Algorithm [NeaHin98]
Generative vs Discriminative Methods [NgJor01]
Discriminative Training of HMMs [McCFrePer00]slides
Discriminative Methods
Feb 2
Conditional Random Fields, Perceptron learning for GM
Chap 2.1,3 Introduction to SVMs [Cristianini, Shaw-Taylor2000]
[LafMcCPer01],[LafZhuLiu04],[Collins02]
Feb 9
SVM based approaches for GM
Chap 6 Introduction to SVMs [Cristianini, Shaw-Taylor2000]
[TsoJoaHofAlt05],[TasGueKol04],[McAllester06]
Feb 16
Boosting based approaches for GM
Tutorial on Boosting [FreSch99]
[AltHofJoh03],[TorMurFre05],[Collins04]
Feb 23
Decompositional approaches
[RotYih05],[LeCHua05],[WesChaEliSchVap02]
Mar 2
Semi-supervised/Unsupervised Learning
[DauMar05], [AltMcCBel05], [BreSch06], [XuWilSouSch06]
Mar 9
Project Presentations