Investigating Gradient Interference and Negative Transfer in Multi-Task Deep Learning

Van-Dinh Nguyen (email | all projects)

Summary

Multi-task learning (MTL) enables a single neural network to learn several related tasks simultaneously, such as image segmentation, attribute prediction, and object classification. Although MTL often improves efficiency and generalization, competing objectives from different tasks can produce conflicting gradients during training, resulting in negative transfer and degraded performance. Recent studies suggest that gradient interference is a key reason why jointly learned tasks do not always outperform independently trained models. This project aims to investigate gradient interference in multi-task learning by comparing conventional MTL with several gradient-balancing techniques, such as GradNorm, PCGrad, and Gradient Detachment. Rather than proposing a completely new model, the project focuses on analysing how different optimisation strategies influence learning dynamics, task performance, and feature representations. The outcome will provide practical guidelines on selecting appropriate optimisation strategies for multi-task computer vision problems.

Objectives

  • Study the principles of multi-task learning and negative transfer.
  • Implement a baseline multi-task learning framework.
  • Evaluate different gradient conflict mitigation methods (e.g., GradNorm, PCGrad, Gradient Detachment).
  • Analyse gradient conflicts using cosine similarity and gradient visualisation.
  • Compare performance across multiple tasks and identify when each optimisation strategy is most effective.

Expected Outcomes

  • A comprehensive comparison of gradient optimisation methods.
  • Visualisation of gradient conflicts during training.
  • Analysis of how gradient interference affects learning and model performance.
  • Recommendations for choosing suitable optimisation strategies in multi-task learning.
  • A reproducible experimental framework that can be extended to other multi-task vision applications.

Key Requirements. Students should have:

  • Good Python programming skills.
  • Experience with PyTorch or TensorFlow.
  • Basic knowledge of convolutional neural networks and deep learning.
  • Understanding of optimisation and backpropagation.
  • Willingness to conduct experiments, analyse results, and interpret visualisations.