31210: Advanced Natural Language Processing

This is the course webpage for the Spring 2019 version of TTIC 31210: Advanced Natural Language Processing.
For the Spring 2017 course, go here.

Quarter: Spring 2019
Time: Monday/Wednesday 1:30-2:50pm
Location: Room 526 (fifth floor), TTIC

Instructor: Kevin Gimpel
Instructor Office Hours: Mondays 2:50-3:15pm, Wednesdays 2:50-4pm, Room 531
Teaching Assistant: Mingda Chen
Teaching Assistant Office Hours: Mondays 3-4pm, TTIC Library (fourth floor)

Prerequisites: TTIC 31190 or permission of the instructor.

Contents:
Textbooks
Grading
Topics
Collaboration Policy
Lateness Policy

Textbooks
All textbooks are optional. We will post optional readings from the following texts to accompany lecture slides.

SLP2: Daniel Jurafsky and James H. Martin. Speech and Language Processing (2nd Edition). Pearson: Prentice Hall. 2009.
SLP3: Drafts of some chapters of the 3rd edition are freely available online.

NNM4NLP: Yoav Goldberg. Neural Network Methods for Natural Language Processing. 2017.
Two copies are on reserve in the TTIC library. An earlier draft is freely available online.

BANLP: Shay Cohen. Bayesian Analysis in Natural Language Processing. 2016.

Grading
5 assignments (15% each)
class participation, including in-class handouts and quizzes (25%)

Topics

Introduction and Brief Review of TTIC 31190 (1 lecture)
Deep Learning for NLP (5 lectures): similarity functions, loss functions, contextualized word embeddings, subword units, sequence-to-sequence models, attention, self-attention, neural machine translation, other applications
Structured Prediction in NLP (4 lectures): sequence labeling, syntactic and semantic parsing, dynamic programming
Generative Modeling (2 lectures): generative models, latent variables, unsupervised learning, variational autoencoders
Bayesian NLP (2 lectures): priors, Bayesian inference, topic modeling, Gibbs sampling
Bayesian Nonparametrics in NLP (2 lectures)
Review and Other Topics (1 lecture)

Collaboration Policy
You are welcome to discuss assignments with others in the course, but solutions and code must be written individually.

Lateness Policy
If you turn in an assignment late, a penalty will be assessed. The penalty will be 2% (of the entire point total) per hour late.

You will have 4 late days to use as you wish during the quarter. Late days must be used in whole increments (i.e., if you turn in an assignment 6 hours late and want to use a late day to avoid penalty, it will cost an entire late day to do so).