Eligibility: this project is only available to students on the online MSc Statistics and Data Science programme. If interested please contact Cangxiong Chen at Cangxiong.Chen@tcd.ie (personal webpage https://cangxiongchen.github.io/).
Modern neural networks are usually overparameterised, i.e. they contain more parameters than necessary to fit the training data. The additional capacity from overparameterisation can improve optimisation and predictive performance, but it also increases the computational cost for training and inference. Compression techniques such as structured pruning and quantisation can reduce the size and computational cost of a model, but they can also reduce its accuracy and incur significant computational cost. Therefore, we should balance the compression-accuracy trade-off and consider energy cost for the full life-cycle of the model. This project will study whether wider neural networks provide enough predictive benefit to justify their additional computational cost and carbon footprint. The student will train small MLPs, CNNs or Transformers on one public environmental, energy-demand or transport dataset, varying network width and comparing the full model with one compression strategy. They will measure predictive performance, model size, runtime, training energy, inference energy and estimated carbon emissions. The project will focus on one dataset, one architecture family and one compression method.