As AI model developers increasingly publish benchmark results alongside new model releases, it is becoming difficult to systematically track which benchmarks are being used and how benchmarking practices change over time. In this project, you will build a tool which extracts all benchmark names from AI research lab model releases (model cards and/or press release) and compile them into structured dataset. The dataset should include relevant metadata e.g., link to the dataset source on Hugging Face, link to paper for original publication of the benchmark and be presented in a dashboard or other visualization. The tool should allow new models to be added easily via a url. The resulting resource will provide a foundation for tracking and understanding benchmarking practices across model developers.
School of Computer Science and Statistics