How do two people coordinate their movement, gaze, gestures and speech during a conversation, and what can these behaviours tell us about the social interaction taking place?
In this open-ended project, the student will use Meta’s Seamless Interaction dataset, a large-scale multimodal dataset of face-to-face interactions containing processed body motion, facial behaviour, gaze, audio, speech and conversational annotations. The student will select a focused research question and use data analysis and/or machine learning to investigate patterns in human conversational behaviour.
Possible questions include: How do speakers and listeners move differently? Can body movement predict when someone is about to take a conversational turn? Who leads and who follows during an interaction? Do conversational partners synchronise their movements? Can personality or interpersonal stance be predicted from nonverbal behaviour? Which combinations of gaze, gesture, posture and facial behaviour indicate engagement?
The project can focus on one behavioural modality or investigate how multiple cues interact, with the aim of identifying nonverbal signals that could ultimately be used to create more natural and socially responsive virtual humans.
The project combines human behaviour analysis, machine learning, multimodal data and social interaction, and would suit a student interested in AI, virtual humans, computer vision or human perception.