This is the lectures page for the Spring 2019 course webpage for TTIC 31210: Advanced Natural Language Processing.
Monday, April 1, 2019:
Lecture 1: Introduction
readings: review course material of
TTIC 31190
Wednesday, April 3, 2019:
Lecture 2: Elements of Neural NLP
readings:
Yoav Goldberg's primer;
Kyunghyun Cho's notes;
Chapter 7 and
Section 9.1 of
SLP3
Monday, April 8, 2019:
Lecture 3: Loss Functions and Word Embeddings
readings:
Collobert et al. (2011),
Mikolov et al. (2013a),
Mikolov et al. (2013b)
Wednesday, April 10, 2019:
Lecture 4: Subword Modeling and Contextualized Word Embeddings
readings:
Schütze (1993),
Huang et al. (2013),
dos Santos and Zadrozny (2014),
Ling et al., (2015),
Kim et al. (2016)
Monday, April 15, 2019:
Lecture 5: Contextualized Embeddings, Encoders, Attention, Transformers
readings:
Peters et al. (2018),
Section 12 of Goldberg,
Vaswani et al. (2017),
The Annotated Transformer,
The Illustrated Transformer
Wednesday, April 17, 2019:
Lecture 6: Finish Transformers; Sequence-to-Sequence Models and Attention
readings:
Devlin et al. (2018),
Sutskever et al. (2014),
Bahdanau et al. (2015)
Monday, April 22, 2019:
Lecture 7: Structured Prediction 1
readings:
Section 9.2 of
SLP3
Wednesday, April 24, 2019:
Lecture 8: Structured Prediction 2
readings:
Chapter 13 and
Appendix A of
SLP3
Monday, April 29, 2019:
Lecture 9: Inference in Structured Prediction
Wednesday, May 1, 2019:
Lecture 10: Inference and Learning in Structured Prediction
Monday, May 6, 2019:
Lecture 11: Finish Learning in Structured Prediction, Start Latent Variable Modeling
readings:
Eisner (2016),
Gimpel and Smith (2010)
Wednesday, May 8, 2019: Lecture 12: Deep Latent-Variable Models for Natural Language (guest lecture by
Sam Wiseman)
readings:
Tutorial Materials
Monday, May 13, 2019:
Lecture 13: Introduction to Bayesian NLP
readings: Chapter 1, Chapter 2, and Section 3.2 of
Cohen (2016),
Jupyter Notebook
Wednesday, May 15, 2019: No class (instructor out sick)
Monday, May 20, 2019:
Lecture 14: Inference in Bayesian NLP
readings: Chapter 3 and Chapter 5 (up to 5.3) of
Cohen (2016)
Wednesday, May 22, 2019:
Lecture 15: Finish Bayesian Inference; Start Bayesian Nonparametrics in NLP
readings: Sections 5.3, 7.1, and 7.2 of
Cohen (2016)
Monday, May 27, 2019: No Class (Memorial Day)
Wednesday, May 29, 2019:
Lecture 16: Finish Bayesian Nonparametrics; Research Advice
readings:
Goldwater et al. (2009);
Creating Effective Slides (video),
Online resources from Doumont
Monday, June 3, 2019: No Class (Instructor Away)
Wednesday, June 5, 2019: No Class (Instructor Away)
Note to those interested in reusing these course materials: Please do so! No attribution needed. PowerPoint files are also posted; just change the extension from pdf to pptx.