Date
|
Topic
|
Reading Due
|
Assignment
|
8/30 | Introduction to class | ||
9/1 | Syllabus, Introduction | The Discipline of Machine Learning (6 pagse)
Sections 1 - 1.3 of Sutton and Barto (7 pages) | |
9/3 | Decision Tree Learning | ||
9/6 | More Decision Tree Learning
Least Mean Squares | ||
9/8 | Perceptrons and Artifical Neural Networks | ||
9/10 | Artifical Neural Networks | ||
9/13 | Bayes Theorem | ||
9/15 | Naive Bayes | ||
9/17 | Bayes Nets | ||
9/20 | Bayes Nets + Intro to COLT | ||
9/22 | Version Spaces and PAC | Reading Response on Naive Bayes due (on Moodle) | |
9/24 | (In)finite Hypothesis Spaces | Reading Response on Neural Nets due (on Moodle) | |
9/27 | Sutton + Barto, Chapter 2: Gambling! | ||
9/29 | S&B, Chapter 3: The RL framework | ||
10/1 | Bellman Equations | ||
10/4 | Policy Evaluation, Policy Iteration | ||
10/6 | Value Iteration | Reading Response: S&B sections 2.1, 2.2, 2.4, 2.5, 2.7, 2.8, 2.11. Due by 6am. | |
10/8 | S&B, Chapter 5: Monte Carlo | ||
10/11 | No class -- Fall Break
| ||
10/13 | More Monte Carlo: Policy Evaluations | ||
10/15 | Offline MC and TD | S&B Chapter 5. Due 6am Monday 10/18. | |
10/18 | Sarsa | ||
10/20 | Q-learning | ||
10/22 | Discussion of project 0, project 1, and Keepaway | Project 1, Step 0. Install rl-competition code. Hack the getAction() function in agents/marioAgentJava/src/edu/rutgers/rl3/comp/ExMarioAgent.java to always go right and send me the code of this function by 6am Friday. | |
10/25 | Eligibility Traces | S&B Chapter 6. Due 6am Wednesday 10/18. Instead of a summary, you can answer the following two questions: "How would you explain the difference between Dynamic Programming, Monte Carlo, and Temporal Difference Learning? When would you use one of these methods instead of another?" | Project 1, checkpoint #1 due |
10/27 | Eligibility Traces -- on the board | ||
10/29 | More Eligibility | Response Due Monday, 6am: Read 7.0, 7.1, 7.2, 7.3, 7.5, 7.8, and 7.9 | |
11/1 | Function Approximation | Project 1, checkpoint #2 due | |
11/3 | Function Approximation 2, The Remix | Read your section for Friday | |
11/5 | Finishing off Function Approximation and Discussion of paper | ||
11/8 | Planning and Learning | ||
11/10 | Planning and Learning, rest of chapter | Skim S&B chapter 8, read section 8.4 (no response due -- project on Monday!) | |
11/13 | RMax and RL-DT | ||
11/15 | Shaping and TAMER | ||
11/17 | More Shaping, X2 | 6am, Monday the 22nd: Send Matt a proposal for Project 3. Suggestions: One of the topics mentioned in class (see 11/17 slides), using some of the UCI data and decision trees, or one of the RL topics in Mario, Tetris, or Mountain Car. Or anything you think of! Also, send Matt suggestions for what you'd find most interesting to discuss in the final 2 weeks of class. | |
11/19 | Guest Lecture: GAs | ||
11/22 | Ensemble Methods | ||
11/24 |
No class -- Thanksgiving Break
| ||
11/26 |
No class -- Thanksgiving Break
| ||
11/29 | Instance-Based Methods | ||
12/1 | No lecture -- Matt was sick | ||
12/3 | Hierarchical RL | ||
12/6 | Transfer Learning | ||
12/8 | Discussion of Intrinsic Rewards | ||
12/10 | Discussion of Helicopter Flight | ||