Human imagination allows people both to relive past events and to simulate future possibilities, in some sense, to see the world as it is not [1]. In fact, the ability to simulate events in our minds is also crucial to reliving past events allowing some elements of the event to be memorized and gaps to be ‘filled in’ during the remembering process by imagination/simulation. Reliving (aka replaying) past memories may also be important to human ability to learn continually by allowing us to refresh what we have previously learnt while learning new things.
Reinforcement learning (RL) and especially deep reinforcement use an approach to learning to perform tasks based on ’trial and error’ in which agents learn by interacting with their environment. Most RL algorithms assume that the environment is stationary, i.e., that its behaviour doesn’t change so that the learnt strategy remains optimal over time. If the environment changes after a strategy is learnt or a new task needs to be learnt, the agent has to resume learning. Unfortunately, learning a new strategy may result in a phenomenon called ‘catastrophic forgetting’ in which the agent forgets what it has previously learnt because the relevant memories are overwritten by the new strategy. Replay may offer a solution to this problem and this project will assess its ability to allow an agent to retain the ability to learn and retain a variety of diverse tasks over time.
[1] Max Bennett (2023). “A Brief History of Intelligence: Humans, AI, and the Five Breakthroughs That Made Our Brains”. Harper Collins.