Mathematical Toolkit - Autumn 2015


TTIC 31150/CMSC 31150

M W 9 - 10:20, TTIC Room 530

Instructor: Madhur Tulsiani

TA: Haris Angelidakis

Office hours: M 3-4 pm (Madhur), W 3-4 pm (Haris)



 

The course is aimed at first-year graduate students and advanced undergraduates. The goal of the course is to collect and present important mathematical tools used in different areas of computer science. The course will mostly focus on linear algebra and probability. We intend to cover the following topics and examples:

  • Abstract linear algebra: vector spaces, linear transformations, Hilbert spaces, inner product, Gram-Schmidt orthogonalization, eigenvalues and eigenvectors, SVD.
  • Least squares, iterative solvers for systems of linear equations, spectral graph theory, perturbation and stability of eigenvalues, basic of tensors.
  • Discrete probability: random variables, Markov, Chebyshev and Chernoff-Hoeffding bounds, Martingales (time permitting), Chernoff bounds for matrices (time permitting).
  • Gaussian variables, concentration inequalities, dimension reduction.
  • Spectral partiotioning and clustering, spectral techniques for semi-random partioning problems.

The course will have 4 homeworks (40 percent), 2 quizzes (10 percent), a midterm (20 percent) and a final (30 percent).


There is no textbook for this course. Useful references for each topic will be listed on this page as the course proceeds.



Homeworks and Announcements

  • Homework 1 (Due on 10/22/15).
  • Homework 2 (Due on 11/06/15).
  • The Midterm will be held in the class on Monday, November 9.
  • There will be an extra class on Friday, November 13 (9 - 10:20 am) to make up for the missed lecture of October 12.
  • Homework 3 (Due on 11/25/15).
  • Quiz during the first 20 minutes of class on Monday, November 23.
  • Homework 4 (Due on 12/06/15).
  • Class canceled on Monday, November 30.


Lecture Plan and Notes

  • 9/28: Fields and vector spaces.
    [Notes]
  • 9/30: Bases of vector spaces, Lagrange interpolation, linear transformations.
    [Notes]
  • 10/5: Eigenvalues and eigenvectors, inner product spaces, orthogonalization.
    [Notes]
  • 10/7: Parseval's identity, adjoint of an operator, Riesz representation theorem, spectral theorem for self-adjoint operators.
    [Notes]
    Also see these great notes by Paul Garrett for the spectral theorem in Hilbert spaces.
  • 10/14: Rayleigh quotients, min-max characterizations of eigenvalues, singular values of a linear transformation.
    [Notes]
  • 10/19: Singular value decomposition, applications to low-rank approximation of matrices.
    [Notes]
  • 10/21: Approximating point sets by low-dimensional subspaces, Gershgorin disc theorem, application to proving bounds for the restricted isometry property.
    [Notes]
  • 10/26: Solving systems of linear equations, Gaussian elimination, iterative methods for sparse systems, gradient descent.
    [Notes]
  • 10/28: Conjugate gradient method for solving sparse systems, Krylov subspaces and the Lanczos method.
    [Notes]
  • 11/2: Lanczos method wrap-up. Basics of probability and random variables.
    [Notes]
  • 11/4: Schwartz-Zippel lemma, polynomial identity testing, expectations of random variables, coupon collection.
    [Notes]
  • 11/11: Markov's and Chebyshev's inequalities, thresholds in random graphs.
    [Notes]
  • 11/13: Threshold phenomena wrap-up, Chernoff-Hoeffding bounds, randomized load balancing.
    [Notes]
  • 11/16: The power of two choices, Introduction to martingales, the Doob martingale, Azuma's inequality.
    [Notes]
  • 11/18: Proof and applications of Azuma's inequality.
    [Notes]
  • 11/23: Concentration for the chromatic number of random graphs.
  • 11/25: Gaussian random variables, Johnson-Lindenstrauss lemma.
    [Notes]
  • 11/30: CANCELED.
  • 12/2: Spectral partitioning, Cheeger's inequality.


Resources



Recommended Books

There is no textbook for this course. The following books may be useful as references:
  • The Linear Algebra a Beginning Graduate Student Ought to Know
    (by Jonathan S. Golan)
  • Foundations of Data Science (by Hopcroft and Kannan)
  • Probability and Computing (by Mitzenmacher and Upfal)
  • The Probabilistic Method (by Alon and Spencer)