User Engagement in Human-Robot Interaction: A Holistic Study

Abstract

In recent years, companion social robots have attracted increasing attention for their potential to support humans across diverse settings, including healthcare and education. To serve effectively, these robots must adapt their interactions to be engaging and likable, which requires the ability to interpret and reason about different forms of user engagement, such as affective engagement and behavioral engagement. In this thesis, engagement is examined through a proposed human-robot interaction (HRI) framework, where a cognitively demanding task is treated as a central element of the interaction. Within this framework, maintaining user focus on the task with minimal distraction is essential for successful task completion. By examining the interplay between robot feedback, user performance, and engagement, this work demonstrates how robots can dynamically adjust their behavior to sustain user engagement without distracting them from the primary task. Here, user engagement is defined as the quality and dynamics of user involvement. Using controlled experimental studies, findings from the research reported in the thesis revealed a fundamental trade-off: affective-based feedback fostered stronger social bonds, whereas performance-based feedback improved user performance. Further, the study developed a deep learning model trained on annotated interaction data to automatically detect engagement states. Extending beyond HRI, the same approach was applied to driver monitoring, where behavioral and physiological markers indicated impaired engagement during intoxication. Ultimately, this research advances our understanding of how social robots can act as effective assistants in cognitively demanding tasks, particularly in therapeutic or assistive contexts. The results contribute not only to the design of more responsive and emotionally intelligent social robots but also open pathways for generalizing the HRI framework to other domains of human-technology interaction.