Empathetic Robotic Companion for Autistic Adolescents with Multimodal Human-Robot Interaction
Open AccessThe advanced techniques for Socially Assistive Robots (SARs) in the field of Human-Robot Interaction (HRI) have attracted considerable attention in recent years. SARs are designed to provide assistance for humans, such as service or companionship, through social interaction. These robots' capabilities to provide care at home for the elderly, service in a restaurant setting, tutoring for education, and intervention for Autism Spectrum Disorder (ASD) have been increasingly investigated. However, a significant challenge in advancing the effect of robotic assistance and intervention in real-world scenarios is the deficiency of robotic capabilities to learn various interaction skills and social contexts adaptively over time, which is the developmental nature of humans. In particular, the application of ASD interventions with that capacity can benefit autistic individuals in terms of mental health, emotional regulation, and social skills development.This dissertation aims to develop a socially assistive robotic companion that is perceptive of personal emotional states, capable of learning social skills over time, and emotionally interactive in order to provide the socio-emotional long-term HRI and intervention for adolescents with ASD that can benefit their mental health by alleviating anxiety and depression. In doing so, we conducted the study to develop a multimodal emotion recognition framework that can adapt to personal developments over time. This emotion recognition framework consists of cutting-edge deep learning models and novel multimodal fusion strategies. This framework analyzed multimodal signals from the HRI and detected emotional cues from the user's verbal and non-verbal behaviors. In addition, a social-behavior module for recognizing and learning the user's behaviors was designed, and a social game was introduced to provide an interactive social experience during HRI. This module focuses on training the robot to adaptively learn social gestures and associate the gestures with the contexts embedded in dynamic social interactions and communications. We also designed an active emotional interaction agent to provide proactive emotional guidance for the user. The decision-making approach for the agent is implemented through reinforcement learning. This agent intends to stimulate a user's emotional state and elicit positive long-term psychological outcomes.We have developed an autonomous robotic system to provide an intervention program we designed, and conducted a user study on adolescents with ASD. The intervention program is designed based on a modularized intervention scheme that aims to provide positive psychological effects for adolescents with ASD. Our previous proposed studies contribute to the realization of each module in our provided intervention. Besides the impact on mental health, our goal is to design a social robotic framework that can learn and interact over long periods of time, promoting mutual growth with the human user. Both typically developing and ASD adolescents benefit from this robot-assisted intervention program with regard to assistance in the neurodevelopmental process and mental health.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.