Balancing competing objectives, eg speed vs. safety, or efficiency vs. fairness, is a core challenge in multi-objective reinforcement learning (MORL). Prior work in my research group has shown how to optimize multiple objectives simultaneously without collapsing them into a single hand-weighted reward. But specifying the right trade-off between objectives numerically is hard: people often can’t state “I want 70% weight on speed and 30% on safety,” yet they can reliably tell which of two behaviours they prefer when they see them side by side. This project explores fine-tuning MORL agents directly from human feedback — using preference pairs or trajectory comparisons rather than predefined reward weights — to learn the trade-offs people actually want. You’ll build on an existing MORL framework, design a mechanism for collecting and learning from human preference judgments (pairwise comparisons or ranked trajectories), and evaluate whether preference-tuned policies better match what people actually want compared to policies trained with fixed, hand-specified objective weights.
Note: suitably for MSc dissertations only (not 4th year projects) and requires prior RL experience. Please describe your experience when emailing me.
References:
Juan C. Rosero, Ivana Dusparic, and Nicolás Cardozo. “Multi-Objective Deep Reinforcement Learning for Optimisation in Autonomous Systems.” arXiv:2408.01188, 2024.
Chenyang Zhao, Vinny Cahill, and Ivana Dusparic. “Balancing Multiple Objectives in Urban Traffic Control with Reinforcement Learning from AI Feedback.” 21st International Conference on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2026).