Metodo

International Studies in Phenomenology and Philosophy

Book | Chapter

191408

Multimodal interfaces and sensory fusion in VR for social interactions

Esubalew Bekele Joshua W. Wade Dayi Bian Lian Zhang Zhi Zheng Amy Swanson Medha Sarkar Zachary Warren Nilanjan Sarkar

pp. 14-24

Abstract

Difficulties in social interaction, verbal and non-verbal communications as well as repetitive and atypical patterns of behavior, are typical characteristics of Autism spectrum disorders (ASD). Advances in computer and robotic technology are enabling assistive technologies for intervention in psychiatric disorders such as autism spectrum disorders (ASD) and schizophrenia (SZ). A number of research studies indicate that many children with ASD prefer technology and this preference can be explored to develop systems that may alleviate several challenges of traditional treatment and intervention. The current work presents development of an adaptive virtual reality-based social interaction platform for children with ASD. It is hypothesized that endowing a technological system that can detect the feeling and mental state of the child and adapt its interaction accordingly is of great importance in assisting and individualizing traditional intervention approaches. The proposed system employs sensors such as eye trackers and physiological signal monitors and models the context relevant psychological state of the child from combination of these sensors. Preliminary affect recognition results indicate that psychological states could be determined from peripheral physiological signals and together with other modalities including gaze and performance of the participant, it is viable to adapt and individualize VR-based intervention paradigms.

Publication details

Published in:

Shumaker Randall, Lackey Stephanie (2014) Virtual, augmented and mixed reality. designing and developing virtual and augmented environments: Designing and developing virtual and augmented environments. Dordrecht, Springer.

Pages: 14-24

DOI: 10.1007/978-3-319-07458-0_2

Full citation:

Bekele Esubalew, Wade Joshua W., Bian Dayi, Zhang Lian, Zheng Zhi, Swanson Amy, Sarkar Medha, Warren Zachary, Sarkar Nilanjan (2014) „Multimodal interfaces and sensory fusion in VR for social interactions“, In: R. Shumaker & S. Lackey (eds.), Virtual, augmented and mixed reality. designing and developing virtual and augmented environments, Dordrecht, Springer, 14–24.