Summer Course on Probability and Machine Learning
TTI-Japan | 2018
This is a short (1 week) course aimed towards a formal introduction to the basics of probability theory and machine learning. The goal of the first half of the course is to make us comfortable and rigorously reason about probability arising in the context of computer science and machine learning. In the second half of the course, we will start with basics of Machine Learning and build our way up to Neural Networks.
Course Plan
Day 1: PyTorch Setup and Introduction to Probability
- Python, PyTorch and NumPy setup
- Definitions, Events, Conditional Probability, Chain Rule, Bayes Rule, Independent Events
- Random Variables, Expectation, Linearity of Expectation, Independent Random Variables, Markov Inequality
Day 2: Concentration Inequalities and MLE
- Coin Toss
- Maximum Likelihood Estimator for Biased Coin Tosses and Dice Rolls
- How to Differentiate between two Biased Coins?
Day 3: Generalization Error and Introduction to ML
- Please Validate your Hypothesis
- How many Samples do You Need to Learn Bias?
- Machine Learning: Motivation and Introduction
- Naive Bayes
- slides
Day 4: Regression and Neural Networks
- Logistic Regression
- Optimization and Model Selection
- Feedforward Neural Networks
- Backpropagation
- slides
Day 5: Advanced Neural Networks
- Convolutional Neural Networks
- Coding Session
- slides
Resources
Chapter 14 of Great Ideas in Theoretical Computer Science by Anil Ada and Klaus Sutner
Lectures 10-17 of Mathematical Toolkit by Madhur Tulsiani
Lecture 6 of Introduction to the Theory of Machine Learning by Avrim Blum
TTIC Machine Learning Summer School by Suriya Gunasekar, Karl Stratos and Mesrob Ohannessian
Deep Learning Textbook by Ian Goodfellow, Yoshua Bengio and Aaron Courville
Day 1
Day 2
MLE Codes
Please find the code we used for MLE for coins here. As a guideline for the homework, we are also uploading the code for estimating bias of a 6-sided dice here.Toy Example | Stock Market
The goal of this homework is to predict the movement of a dummy stock-market. You will be given data points: . The data point corresponds to the movement of the stock-market on day. Assume that there are working hours, and corresponds to whether the market went up or down during hour on day. That is, if market went up, and if it went down.
Assume that market movements on different days are i.i.d. You may not assume anything about dependence of the market movement during a single day. For example, it may happen that if market went up in first hour, it will surely go down in the last hour irrespective of what happened in the middle three hours.
You can find the data for 10,000 days of market movement here. Your goal is to predict market movement on the last hour for the 10 days of data present here.
We will discuss the solution tomorrow.