CSC 5661 - Reinforcement Learning

4 lecture hours 0 lab hours 4 credits

Course Description
At the most basic level an artificial intelligence (AI) is an automated decision-making system. In order to maximize the potential of AI, we need methods to teach it to make "good" decisions over long periods of time. Reinforcement learning (RL) is one such method. This course focuses on building the expertise needed to construct and train AIs using RL. We will learn about key concepts, mathematics, algorithms, and applications of RL. We will begin by introducing different types of RL problems and the algorithms available to solve them. Next we will introduce Markov decision processes which will give the agents we seek to develop a formal framework that they can use to solve problems. Following this, students will review and implement multiple classical and modern foundational RL algorithms to solve applied AI control problems. Environmental simulators, performance metrics, on/off policy algorithms, the exploration vs exploitation tradeoff, and state/action/reward engineering will be discussed. Lecture content will be reinforced with hands-on, tutorial-oriented laboratory exercises for development of representative applications. Students will finish the course by designing, implementing, testing, and documenting an AI trained using RL on a problem of their choice.
Prereq: (MTH 2480 and MTH 2340) or MTH 5810 and (CSC 2621 or CSC 5610) or instructor consent
Note: This course is open to qualified undergraduate students.

Course Learning Outcomes
Upon successful completion of this course, the student will be able to:

Define key features of reinforcement learning (RL) that distinguish it from other forms of machine learning
Take a given substantive problem (e.g. from computer vision, robotics, etc), decide if it should be formulated as a RL problem
Define a RL problem it formally (in terms of the state space, action space, dynamics, and reward model)
Given a RL problem determine what algorithm (from class) is best suited for addressing it
Implement environmental simulators capable of generating the data needed to train an RL agent in Python
Implement the common RL algorithms and their underlying component processes in Python
Describe (list and define) multiple criteria for analyzing RL algorithms and evaluate algorithms on these metrics: e.g. regret, sample complexity, computational complexity, empirical performance, convergence, etc.
Describe the exploration vs exploitation challenge and compare and contrast at least two approaches for addressing this challenge (in terms of performance, scalability, complexity of implementation, and theoretical guarantees)

Prerequisites by Topic

Calculus
Linear algebra
Machine learning experimental design (e.g., as encountered in CSC 2621 or CSC 5610)
Some previous exposure to deep learning
Python (e.g. experience with numpy and object oriented programming)

Coordinator
Dr. Jeremy Kedziora

Add to Portfolio.

Close Window