VR-based Biofeedback Systems for Enhancing Social and Emotional Competence in Autism – Dr. Joy Li


This continuous research project aims to look for effective intervention or therapy methods using cutting edge VR devices and non-intrusive gaming methods to help mental healthcare. The basic idea is to let autistic children play serious games that dynamically adapt to their emotions while their emotional reactions are recorded by several devices, and their emotions are analyzed by physiological signals. The adaptive system provides evaluation for serious game-based therapy sessions, as well as the potential directions for further treatment. An affective game framework that targets the intervention of autism will be developed, using the most advanced VR tools but not limited to VR. This framework is a new approach that integrates different layers of emotion analyzers such as behavioral analysis including audio emotion detection, facial emotion detection, and textual emotion detection, as well as physiological signal emotion analysis. With fusion and integration, the game will be able to provide feedback to the current emotional status. Pilot data from experiments show significant differences in emotional re-activities between the autistic groups and the control group, which proved the assumption that serious games could be used in intervention or therapy for the autistic group, especially targeting their emotional competence. This research is looking to seek an NIH grant for 2020.