Testing software means systematically probing it for the edge cases that break it — but RL policies don’t come with a specification to test against, and their failures often only surface deep into a long trajectory, making them hard to catch before deployment. Counterfactual explanation techniques developed in this group already show that tracing “what minimal change would have avoided this failure” is a powerful way to diagnose why a trained RL policy went wrong. This project reframes that idea through a software-testing lens: rather than explaining failures after the fact, one at a time, you’ll build a framework that proactively and systematically searches for a diverse set of failure-inducing scenarios across a trained policy’s behaviour space — effectively generating a targeted “test suite,” evaluated using software-testing-style metrics (failure diversity, state-space coverage, minimality of each test case) rather than the human-interpretability metrics typically used in explainability research. You’ll compare this counterfactual-guided search against standard test-generation baselines from the software testing literature (e.g., random fuzzing, adversarial state perturbation) to assess whether it finds a more informative and representative set of failure cases.
Note on novelty: using counterfactual explanations to diagnose individual RL failures is established in prior work (see references below). The contribution of this project is not a new explanation mechanism, but a systematic, test-suite-generation framing and evaluation protocol — applying and adapting these existing techniques through a software-engineering testing lens rather than an explainability lens.
References:
Jasmina Gajcin and Ivana Dusparic. “ACTER: Diverse and Actionable Counterfactual Sequences for Explaining and Diagnosing RL Policies.” arXiv:2402.06503, 2024.
Jasmina Gajcin and Ivana Dusparic. “RACCER: Towards Reachable and Certain Counterfactual Explanations for Reinforcement Learning.” 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2024).
Jasmina Gajcin and Ivana Dusparic. “Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunities.” ACM Computing Surveys, 56(9), Article 219, 2024.
Stratis Tsirtsis, Abir De, and Manuel Gomez-Rodriguez. “Counterfactual Explanations in Sequential Decision Making Under Uncertainty.” NeurIPS 2021. (Related work: k-action counterfactual sequences in finite-horizon MDPs.)
Related work on debugging framing: ACM Transactions on Intelligent Systems and Technology (2025), “Counterfactual Explainer for Deep Reinforcement Learning Models using Policy Distillation” — discusses counterfactual explanations for debugging failure cases and corner cases in safety-critical DRL systems.