Information and Coding Theory - Winter 2021


TTIC 31200/CMSC 37220

T Th 1:00-2:20 (Zoom)

Discussion: F 2-3 pm (Gather)

Office hours: W 1-2 pm (Gather)

Instructor: Madhur Tulsiani

TA: Goutham Rajendran



 

This course is meant to serve as an introduction to some basic concepts in information theory and error-correcting codes, and some of their applications in computer science and statistics. We plan to cover the following topics:

  • Introduction to entropy and source coding. Some applications of entropy to counting problems.
  • Mutual information and KL-divergence. Method of types and hypothesis testing. Minimax rate bounds.
  • I-projections, maximum entropy, exponential families and applications.
  • Introduction to error-correcting codes. Unique and list decoding of Reed-Solomon and Reed-Muller codes.
  • Applications of information theory to problems in theoretical computer science.

The course will have 4-5 homeworks (60 percent) and a final (40 percent).


There is no textbook for this course. A useful reference is ``Elements of Information Theory'' by T. M. Cover and J. A. Thomas. Also take a look at the resources section below.


Access information for lectures, discussions and office hours, is available via the Canvas page for the course.



Homeworks and Announcements



Lecture Plan and Notes



Resources