Research


Publications

Conversational Group Detection with Graph Neural Networks
Sydney Thompson, Abhijit Gupta, Anjali W. Gupta, Austin Chen, Marynel Vázquez
23rd ACM International Conference on Multimodal Interaction (ICMI) 2021
We study conversational group detection in varied social scenes using a message-passing Graph Neural Network (GNN) in combination with the Dominant Sets clustering algorithm. Our approach first describes a scene as an interaction graph, where nodes encode individual features and edges encode pairwise relationship data. Then, it uses a GNN to predict pairwise affinity values that represent the likelihood of two people interacting together, and computes non-overlapping group assignments based on these affinities. We evaluate the proposed approach on the Cocktail Party and MatchNMingle datasets. Our results suggest that using GNNs to leverage both individual and relationship features when computing groups is beneficial, especially when more features are available for each individual.
Paper | Code

Challenges Deploying Robots During a Pandemic: An Effort to Fight Social Isolation Among Children
Nathan Tsoi, Joe Connolly, Emmanuel Adéníran, Amanda Hansen, Kaitlynn Taylor Pineda, Timothy Adamson, Sydney Thompson, Rebecca Ramnauth, Marynel Vázquez, Brian Scassellati
ACM/IEEE International Conference on Human-Robot Interaction (HRI) 2021
Best Paper Candidate
The practice of social distancing during the COVID-19 pandemic resulted in billions of people quarantined in their homes. In response, we designed and deployed VectorConnect, a robot teleoperation system intended to help combat the effects of social distancing in children during the pandemic. VectorConnect uses the off-the-shelf Vector robot to allow its users to engage in physical play while being geographically separated. We distributed the system to hundreds of users in a matter of weeks. This paper details the development and deployment of the system, our accomplishments, and the obstacles encountered throughout this process. Also, it provides recommendations to best facilitate similar deployments in the future. We hope that this case study about Human-Robot Interaction practice serves as inspiration to innovate in times of global crises.

Improving Social Awareness Through DANTE: Deep Affinity Network for Clustering Conversational Interactants
Mason Swofford, John Peruzzi, Nathan Tsoi, Sydney Thompson, Roberto Martín-Martín, Silvio Savarese, Marynel Vázquez
Proceedings of the ACM on Human-Computer Interaction
We propose a data-driven approach to detect conversational groups by identifying spatial arrangements typical of these focused social encounters. Our approach uses a novel Deep Affinity Network (DANTE) to predict the likelihood that two individuals in a scene are part of the same conversational group, considering their social context. The predicted pair-wise affinities are then used in a graph clustering framework to identify both small (e.g., dyads) and large groups. The results from our evaluation on multiple, established benchmarks suggest that combining powerful deep learning methods with classical clustering techniques can improve the detection of conversational groups in comparison to prior approaches. Finally, we demonstrate the practicality of our approach in a human-robot interaction scenario. Our efforts show that our work advances group detection not only in theory, but also in practice.
Website | Code | Paper


Projects

Robots For Good: Fighting Social Isolation with Robots
Robotic telepresence for elementary school-aged children enabling socially distant play.



Awards

HRI 2021 Best Paper Award Candidate
For the work: Challenges Deploying Robots During a Pandemic: An Effort to Fight Social Isolation Among Children by N. Tsoi, J. Connolly, E. Adéníran, A. Hansen, K. T. Pineda, T. Adamson, S. Thompson, R. Ramnauth, M. Vázquez, B. Scassellati