This is the 2025 Spirng course on deep learning and reinforcement learning at SUSTech.

Course title: Deep Learning and Reinforcement Learning

Hours/Credits: 32/2

Academic Dept.: School of System Design and Intelligent Manufacturing

Scheduled: Spring Semester

Course type: Elective

Language of Instruction: Chineses and English

Prerequisites: There is no official pre-requisite of the course, although we expect students to have good background in linear algebra, optimization and probability. We will be primarily using Python. As such, I expect you to have elementary programming skills, e.g., writing a hello world program.

This course presents fundamental theory and algorithms about deep learning and reinforcement learning. It is expected that the students can

• master fundamental theory and algorithms about deep learning and reinforcement learning through lectures and literature reading
• grasp skills of deep learning and reinforcement learning with Python language through assignments and projects
• design and implement technical projects, and apply developed techniques to real-world applications (images, signals, robotics, etc.).

Course Assessment

• Form of examination: Class Performance + Mid-term project + Final project + Final exam
• Grading policy: Class performance 10%, Mid-term project 30%, Final project 30%, Final exam 30%.

Textbook and Supplementary Readings:

• R. S. Sutton and A. C. Barto, Reinforcement learning: an introduction (second edition), The MIT Press, 2018
• C. M. Bishop, Pattern Recognition and Machine Learning , Springer, 2006

1. Introduction to deep learning and reinforcement learning
2. PyTorch basics
3. Linear regression
4. Linear classification
5. Fully connected feedforward neural networks (FFNNs)
6. Convolutional neural networks (CNNs)
7. Recurrent neural networks (RNNs)
8. Attention and transformer neural networks
9. Markov decision processes (MDP)
10. Planning by dynamic programming
11. Model-free prediction
12. Model-free control
13. Value function approximation
14. Policy gradient
15. Integrating learning and planning
16. Deep reinforcement learning