Deep Reinforcement Learning

• Be able to decide whether a given application problem should be formulated as a Deep Reinforcement Learning (DRL) problem.
• Be able to correctly define the problem formulation, design the most suitable algorithm from the different possible classes of DRL algorithms, providing a justification.
• Understand the multiple criteria for analysing and evaluating the DRL algorithms on the relevant metrics: regret, sample complexity, computational complexity, empirical performance, convergence.
• Implement in code the main DRL algorithms and apply it to solve several practical problems in different application domains, evaluating experimentally their performance.

This course offers an in-depth study on both theoretical and algorithmic foundations of Deep Reinforcement Learning (DRL). The contents of the course are relevant for applications in multiple domains with complex decision-making tasks, such as robotic control, autonomous vehicles and navigation, resource allocation in wireless networks, optimizing inventory or industrial processes, resource management in computing clusters, traffic control, games, finance, medicine, personalized recommendations, bidding and advertising, web navigation, among others.

The course covers the following main topics:

• Introduction: From supervised learning to decision making, formal framework of reinforcement learning (RL), Markov Decision Process (MDP), motivation, comparison with exact dynamic programming framework, applications in different domains, learnable functions, role of deep learning (DL) for RL, on/off policy methods, overview of schemes and algorithms of Deep Reinforcement Learning (DRL).
• Tutorials and Review of TensorFlow/Pytorch and Neural Nets.
• Policy-based algorithms: Policy-gradient derivation, Stochastic policy gradient, Deterministic policy gradient.
• Value-based algorithms: The Q- and V- Functions, Q-learning, Deep Q-networks.
• Model-based algorithms
• Combined methods: actor-critic methods, trust region optimization and proximal policy optimization, integrating model-free and model-based algorithms.
• Practical details: algorithm implementation, parallelization methods, performance evaluation, choice of hyperparameters, choice of neural network architectures.
• Other advanced topics: connection between RL-based control and inference, exploration methods, inverse reinforcement learning, concept of generalization, transfer learning and multi-task learning, meta-learning.