CptS 580: Reinforcement Learning
Spring 2015
Matthew E. Taylor (Matt)
taylorm@eecs.wsu.edu
EME 137
Syllabus
:
Spring 2015
Textbook
:
Reinforcement Learning: An Introduction
Resources
:
Please use:
Piazza
for reading responses
email to Matt and TAs
for exercise submission
Machine Learning 3 - Reinforcement Learning
Gabe's office hour in Dana 3: Monday, 2-3pm
Bei's office hour in Dana 3: Thursday, 3-4pm
Suggested
exercises
Schedule
Date
Topic
Homework
1/13
First day of class: Introduction
1/15
Secord day of class: Introduction, continued
Read: Chapter 1
Udacity: Sign up for
Machine Learning 3 - Reinforcement Learning
Watch: "Introduction" through "Markov Decision Pocess Four" (8 videos total)
Sign up for
Piazza
1/20
Bandits!
Read: Chapter 2
Udacity: More About Rewards (1 & 2) + Rewards Quiz
Please write a response to both on Piazza by 5pm on Monday
1/22
MDPs
Read up through section 3.5 in the book
Please write a response on Piazza by 5pm on Wednesday
1/27
No class: AAAI
1/29
No class: AAAI
Read up through chapter 4
Finish watching RL 1 - Markov Decision Processes
Please write a response on Piazza by 5pm on Wednesday
2/3
Dynamic Programming
2/5
No class: NSF
2/6
Makeup class: 9:30-11am =
Dynamic Programming
2/10
Monte Carlo Methods
Read up through chapter 5
Watch RL 2: Reinforcement Learning - Three Approaches to RL (5 videos)
Please write a response on Piazza by 5pm on Monday
2/12
Monte Carlo Methods
Watch RL 2: "A New Kind of Value Function" - "Learning Incrementally" (5 videos)
No response required
2/17
Temporal Difference Methods
Read up through chapter 6
Watch RL 2: "Estimating Q From Transitions 2" - "What Have We Learned" (6 videos)
Please write a response on Piazza by 5pm on Monday on Chapter 6 and/or the last 11 videos
2/19
Temporal Difference Methods
2/24
Eligibility Traces
Read chapter 7, write a Piazza response by 5pm on Monday
2/26
Eligibility Traces
3/3
Function Approximation
. Class
video
Read chapter 8, write a Piazza response by 5pm on Monday
3/5
Function Approximation
. Class
video
3/10
Planning and Model Learning
Read chapter 9, write a Piazza response by 5pm on Monday
3/12
Planning and Model Learning
3/17
No class: Spring Break
3/19
No class: Spring Break
3/24
Discussion of RAM-RMAX
Read
Efficient Reinforcement Learning with Relocatable Action Models
and write a response on Piazza by 5pm on Monday.
3/26
Guest Lecture on Multi-agent Learning: Carrie Rebhuhn
3/31
Shaping Rewards
4/2
Intrinsic Motivation in RL
4/7
Transfer Learning
4/9
Leah
1-3 paragraph final project proposal posted to Piazza by 5pm on 4/8
Read paper on Extrinsic/Intrinsic Motivation (see piazza)
4/14
Yang
Please look at this
website and paper
4/16
Zhaodong
4/21
James
4/23
Sal
4/28
Duy
4/30
Final Class
Final Project (Draft) Due
5/7
Final Project Due.
For formatting, I suggest you use the
AAAI-15
style in either latex or Word.
Possible further topics
Current Function Approximation Choices
Efficient Model-Learning methods
Hierarchical Methods
Game Playing
Learning in Robotics
Transfer Learning
Shaping Rewards
Learning from Human Rewards
Learning from Demonstration
Multi-agent RL
Partially observable envirnments and/or POMDPs
Meta-RL and empirical evaluation of algorithms
Least Squares methods (e.g., LSPI)
Adaptive Representations / Representation Learning
Case Studies: Robot soccer, Helicopter Control, etc.
Inverse Reinforcement Learning (IRL)
Intrinsicly Motivated Reinforcement Learning
Actor-Critic Methods
Policy Gradient methods
Crowd Sourcing (?)