Automated tests can sometimes become flaky, passing and failing across executions even when the software has not changed, reducing confidence in Software Testing and Continuous Integration. Recent studies have shown that web test flakiness can arise from factors such as asynchronous DOM interactions, timing assumptions, shared state, and test-order dependencies. The aim of this project is to investigate techniques for detecting and characterising flaky tests in web applications, using existing open-source projects and flaky-test datasets. A direction could be to reproduce and analyse real flaky tests, then develop a prototype detection approach using techniques such as repeated execution, static/dynamic analysis, or execution traces. In general the approach could be evaluated in terms of its ability to detect and explain flaky failures, with automatic repair considered as a possible extension.
References:
[PMcM] https://philmcminn.com/publications/flaky-tests
[1] Y. Pei, J. Sohn and M. Papadakis, “An Empirical Study of Web Flaky Tests: Understanding and Unveiling DOM Event Interaction Challenges” ICST 2025
[2] N. Hashemi, A. Tahir, S. Rasheed, A. Shi and R. Blagojevic, “Detecting and Evaluating Order-Dependent Flaky Tests in JavaScript”, ICST 2025
[3] X. Cai, Z. Dong, Y. Wang, A. Tiwari and X. Peng, “Reproducing Timing-Dependent GUI Flaky Tests in Android Apps via a Single Event Delay”, ISSTA 2024
[4] M. Gruber, M. F. Roslan, O. Parry, F. Scharnböck, P. McMinn and G. Fraser, “Do Automatic Test Generation Tools Generate Flaky Tests?”, ICSE 2024