In cooperative multi-agent reinforcement learning, several agents have to learn how to work together. A challenge is that they can sometimes settle on a reasonably good way of cooperating, even though a much better joint solution exists. This project investigates whether the way learning experiences are selected and used during training could influence which solution the agents eventually discover. In particular, it may be useful to preserve and make better use of rare experiences where the agents successfully coordinate, rather than allowing these experiences to be lost among many less useful ones.