UoE RL Reading Group | 27 June 2024

Speaker: Eduardo Pignatelli (University College London)

Title: On the temporal credit assignment in Deep RL

Abstract: The Credit Assignment Problem (CAP) refers to the longstanding challenge of Reinforcement Learning agents to associate actions with their long-term consequences. Solving the CAP is a crucial step towards the successful deployment of RL in the real world since most decision problems provide feedback that is noisy, delayed, and with little or no information about the causes. These conditions make it hard to distinguish serendipitous outcomes from those caused by informed decision-making. We review the state of the art of Temporal Credit Assignment (CA) in deep RL, and present a unifying formalism for credit, casting the CAP as the problem of learning the influence of an action over an outcome from a finite amount of experience. Finally, we review Credit Assignment with Language Models (CALM), and how language models can assist to bridge the gap between the current state of the art, and the issues identified in the survey.

Link: https://openreview.net/forum?id=bNtr6...

Bio: Eduardo investigates Deep Reinforcement Learning with a focus on the Credit Assignment Problem at UCL, as a PhD student under the supervision of Laura Toni and Tim Rocktäschel. Previously, he was the Machine Learning Lead at BuroHappold Engineering, where he focused on integrating machine automation into the design process. He was a Research Assistant at the Department of Bioengineering of Imperial College London, where he developed a surrogate model for the electrophysiology of the heart to prevent cardiac arrhythmias. Eduardo holds a Master of Science in Architecture from the University of Naples Federico II, where he developed generative models for the design of acoustic shells for outdoor chamber music using genetic algorithms.