Ivana Dusparic

I am open to supervising projects developing novel artificial intelligence techniques and/or applying these techniques in robotics, intelligent urban mobility and smart cities in general. In particular, I am interested in learning-based agents and multi-agent systems, with particular focus on reinforcement learning (RL), including deep reinforcement learning, transfer learning, lifelong learning, credit assignment, multi-agent collaboration, self-adaptive systems, and self-aware systems . I am also interested in applying these techniques to robotics as well as emerging urban transport models and their impact on cities, eg intelligent urban traffic control. I am also open to other proposals applying learning-based multi-agent techniques to management and optimization of other large-scale infrastructures - if you have an interest in learning-based optimization and have an application area in mind, let me know!

reinforcement learning from LLM feedback

Ivana Dusparic (email | all projects)

This project aims to work on techniques for fine tuning LLMs to act as a source of a reward in a reinforcement learning system – whether to replace or complement standard RL rewards, or to act as a source of alignment of an RL process with human preference. The project is suitable for an MSc … Read more

Self-Coordination in Multi-Agent Reinforcement Learning Applied to Railway Domain

Ivana Dusparic (email | all projects)

This project will explore applications of Reinforcement Learning (RL) for a real-world application in the railway domain. To address increasing demand, railway providers aim to increase traffic density on the existing network. However, dense traffic can result in delayed trains and infrastructure disruptions, which can impact planned trips in a large part of the network … Read more

Anything and everything RL

Ivana Dusparic (email | all projects)

Have experience with and your own FYP/MSc project idea related to Reinforcement Learning (“plain” or multi-objective, multi-agent, transfer, lifelong, explainable, inverse, combined with LLMs, etc?) – whether to develop new algorithms or to apply existing ones to a new application area? Contact me with your idea to see if we can formulate the topic together. 

Counterfactual explanations for Explainable and Trustworthy Reinforcement Learning

Ivana Dusparic (email | all projects)

Reinforcement learning (RL) has been successfully applied in a wide range of domains, demonstrating its potential to perform complex tasks by optimizing reward signals obtained through interaction with the environment. However, real-world tasks often involve multiple, potentially conflicting objectives that are not easily represented by a single scalar reward. Multi-Objective Reinforcement Learning (MORL) addresses this … Read more

Cooperative driving using multi-agent RL

Ivana Dusparic (email | all projects)

This project will explore use of and development of new multi-agent RL techniques to achieve cooperation and coordination between autonomous vehicles in order to optimize traffic congestion. Prior experience with RL is essential

Multi-objective optimization in reinforcement learning

Ivana Dusparic (email | all projects)

This project will apply and extend multi-objective (MO) technique Deep W-Networks[1] to a number of benchmark environments from multi-objective gym [2], and benchmark against other MO RL techniques. References: [1] J Hribar, L Hackett, I Dusparic. Deep W-Networks: Solving Multi-Objective Optimisation Problems With Deep Reinforcement Learning. International Conference on Agents and Artificial Intelligence (ICAART) 2023 … Read more

Software testing techniques for reinforcement learning-based systems

Ivana Dusparic (email | all projects)

This project will investigate existing software testing techniques and development of new ones for testing of reinforcement learning-based software applications. References: [1] Yuteng Lu, Weidi Sun, Meng Sun, Towards mutation testing of Reinforcement Learning systems, Journal of Systems Architecture, Volume 131, 2022, https://www.sciencedirect.com/science/article/pii/S1383762122001977 [2] Miller Trujillo, Mario Linares-Vásquez, Camilo Escobar-Velásquez, Ivana Dusparic, and Nicolás Cardozo. … Read more

Explainable/Trustworthy Reinforcement Learning

Ivana Dusparic (email | all projects)

In the recent years causal inference has emerged as an important approach for addressing different issues within RL. Providing agents the ability to leverage causal knowledge was identified as a key ingredient in developing human-centered explanation methods. Namely, when using AI systems, humans tend to be interested in answering questions such as “What caused the … Read more