Recent advances in generative AI make it possible to realistically transform both a person’s face and voice while preserving their underlying behaviour. This project will develop an interactive AI virtual mirror in which participants see their face transformed into another photorealistic identity while their own facial expressions and head movements drive the generated face. The project will build on recent real-time face re-enactment approaches such as LivePortrait and Deep-Live-Cam.
The student will additionally explore emerging AI voice-conversion techniques, allowing the participant’s speech to be transformed into a different voice while preserving what they say and how they speak. This will allow different combinations of facial and vocal identity to be manipulated independently or together.
A perceptual experiment will investigate how these transformations influence self-identification, ownership and agency. Does a synchronously moving face begin to feel like your face even when it looks completely different? What happens when your voice changes too? Does a voice that is congruent with the new appearance strengthen identification with the transformed person?
The student could manipulate factors such as face identity, voice identity, face–voice congruence, expression fidelity and latency to investigate which audiovisual cues are most important for maintaining or altering our sense of self.
The project combines computer vision, speech processing, generative AI, real-time graphics and human perception, and would suit a student taking modules in Real-Time Animation and Computer Vision/Deep Learning, with strong programming skills.