Appl Clin Inform 2018; 09(01): 129-140
DOI: 10.1055/s-0038-1626727
Research Article
Schattauer GmbH Stuttgart

Feasibility Testing of a Wearable Behavioral Aid for Social Learning in Children with Autism

Jena Daniels
Nick Haber
Catalin Voss
Jessey Schwartz
Serena Tamura
Azar Fazel
Aaron Kline
Peter Washington
Jennifer Phillips
Terry Winograd
Carl Feinstein
Dennis P. Wall
Funding The work was supported in part by funds to D.P.W. from NIH (1R01EB025025-01 & 1R21HD091500-01), The Hartwell Foundation, Bill and Melinda Gates Foundation, Coulter Foundation, Lucile Packard Foundation, and program grants from Stanford's Precision Health and Integrated Diagnostics Center (PHIND), Beckman Center, Bio-X Center, Predictives and Diagnostics Accelerator (SPADA) Spectrum, and Child Health Research Institute. We also acknowledge generous support from David Orr, Imma Calvo, Bobby Dekesyer and Peter Sullivan.
Further Information

Publication History

15 November 2017

01 January 2018

Publication Date:
21 February 2018 (online)


Background Recent advances in computer vision and wearable technology have created an opportunity to introduce mobile therapy systems for autism spectrum disorders (ASD) that can respond to the increasing demand for therapeutic interventions; however, feasibility questions must be answered first.

Objective We studied the feasibility of a prototype therapeutic tool for children with ASD using Google Glass, examining whether children with ASD would wear such a device, if providing the emotion classification will improve emotion recognition, and how emotion recognition differs between ASD participants and neurotypical controls (NC).

Methods We ran a controlled laboratory experiment with 43 children: 23 with ASD and 20 NC. Children identified static facial images on a computer screen with one of 7 emotions in 3 successive batches: the first with no information about emotion provided to the child, the second with the correct classification from the Glass labeling the emotion, and the third again without emotion information. We then trained a logistic regression classifier on the emotion confusion matrices generated by the two information-free batches to predict ASD versus NC.

Results All 43 children were comfortable wearing the Glass. ASD and NC participants who completed the computer task with Glass providing audible emotion labeling (n = 33) showed increased accuracies in emotion labeling, and the logistic regression classifier achieved an accuracy of 72.7%. Further analysis suggests that the ability to recognize surprise, fear, and neutrality may distinguish ASD cases from NC.

Conclusion This feasibility study supports the utility of a wearable device for social affective learning in ASD children and demonstrates subtle differences in how ASD and NC children perform on an emotion recognition task.

Protection of Human and Animal Subjects

This study was approved by the Institutional Review Board at Stanford University's School of Medicine, IRB Protocol 31817. Participants' assent and parents' informed consent were received before inclusion in the study.

  • References

  • 1 Buescher AV, Cidav Z, Knapp M, Mandell DS. Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatr 2014; 168 (08) 721-728
  • 2 Christensen DL, Baio J, Van Naarden Braun K. , et al; Centers for Disease Control and Prevention (CDC). Prevalence and characteristics of autism spectrum disorder among children aged 8 years--Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2012. MMWR Surveill Summ 2016; 65 (03) 1-23
  • 3 Christensen DL, Bilder DA, Zahorodny W. , et al. Prevalence and characteristics of autism spectrum disorder among 4-year-old children in the Autism and Developmental Disabilities Monitoring Network. J Dev Behav Pediatr 2016; 37 (01) 1-8
  • 4 Nicholas JS, Charles JM, Carpenter LA, King LB, Jenner W, Spratt EG. Prevalence and characteristics of children with autism-spectrum disorders. Ann Epidemiol 2008; 18 (02) 130-136
  • 5 Robins DL. Prevalence counts: commentary on “prevalence and characteristics of autism spectrum disorder among 4-year-old children in the Autism and Developmental Disabilities Monitoring Network.”. J Dev Behav Pediatr 2016; 37 (01) 80-82
  • 6 Dawson G, Rogers S, Munson J. , et al. Randomized, controlled trial of an intervention for toddlers with autism: the Early Start Denver Model. Pediatrics 2010; 125 (01) e17-e23
  • 7 Howlin P, Goode S, Hutton J, Rutter M. Adult outcome for children with autism. J Child Psychol Psychiatry 2004; 45 (02) 212-229
  • 8 Landa RJ, Holman KC, Garrett-Mayer E. Social and communication development in toddlers with early and later diagnosis of autism spectrum disorders. Arch Gen Psychiatry 2007; 64 (07) 853-864
  • 9 Landa RJ, Holman KC, O'Neill AH, Stuart EA. Intervention targeting development of socially synchronous engagement in toddlers with autism spectrum disorder: a randomized controlled trial. J Child Psychol Psychiatry 2011; 52 (01) 13-21
  • 10 Palumbo L, Burnett HG, Jellema T. Atypical emotional anticipation in high-functioning autism. Mol Autism 2015; 6: 47
  • 11 Sasson NJ, Pinkham AE, Weittenhiller LP, Faso DJ, Simpson C. Context effects on facial affect recognition in schizophrenia and autism: behavioral and eye-tracking evidence. Schizophr Bull 2016; 42 (03) 675-683
  • 12 Xavier J, Vignaud V, Ruggiero R, Bodeau N, Cohen D, Chaby L. A multidimensional approach to the study of emotion recognition in autism spectrum disorders. Front Psychol 2015; 6: 1954
  • 13 Marsh AA, Blair RJR. Deficits in facial affect recognition among antisocial populations: a meta-analysis. Neurosci Biobehav Rev 2008; 32 (03) 454-465
  • 14 Blair RJR. Responding to the emotions of others: dissociating forms of empathy through the study of typical and psychiatric populations. Conscious Cogn 2005; 14 (04) 698-718
  • 15 Rump KM, Giovannelli JL, Minshew NJ, Strauss MS. The development of emotion recognition in individuals with autism. Child Dev 2009; 80 (05) 1434-1447
  • 16 Yirmiya N, Sigman MD, Kasari C, Mundy P. Empathy and cognition in high-functioning children with autism. Child Dev 1992; 63 (01) 150-160
  • 17 Burnette CP, Mundy PC, Meyer JA, Sutton SK, Vaughan AE, Charak D. Weak central coherence and its relations to theory of mind and anxiety in autism. J Autism Dev Disord 2005; 35 (01) 63-73
  • 18 Schreibman L, Dawson G, Stahmer AC. , et al. Naturalistic developmental behavioral interventions: empirically validated treatments for autism spectrum disorder. J Autism Dev Disord 2015; 45 (08) 2411-2428
  • 19 Nash JM. Fertile minds. Time 1997; 149 (05) 48-56
  • 20 Phillips DA, Shonkoff JP. From Neurons to Neighborhoods: The Science of Early Childhood Development. Sydney, Australia: National Academies Press; 2000
  • 21 Dawson G, Bernier R. A quarter century of progress on the early detection and treatment of autism spectrum disorder. Dev Psychopathol 2013; 25 (4 Pt 2): 1455-1472
  • 22 Dawson G, Burner K. Behavioral interventions in children and adolescents with autism spectrum disorder: a review of recent findings. Curr Opin Pediatr 2011; 23 (06) 616-620
  • 23 Dawson G, Webb SJ, McPartland J. Understanding the nature of face processing impairment in autism: insights from behavioral and electrophysiological studies. Dev Neuropsychol 2005; 27 (03) 403-424
  • 24 Chen SH, Bernard-Opitz V. Comparison of personal and computer-assisted instruction for children with autism. Ment Retard 1993; 31 (06) 368-376
  • 25 Golan O, Ashwin E, Granader Y. , et al. Enhancing emotion recognition in children with autism spectrum conditions: an intervention using animated vehicles with real emotional faces. J Autism Dev Disord 2010; 40 (03) 269-279
  • 26 Grynszpan O, Weiss PL, Perez-Diaz F, Gal E. Innovative technology-based interventions for autism spectrum disorders: a meta-analysis. Autism 2014; 18 (04) 346-361
  • 27 Mitchell P, Parsons S, Leonard A. Using virtual environments for teaching social understanding to 6 adolescents with autistic spectrum disorders. J Autism Dev Disord 2007; 37 (03) 589-600
  • 28 Moore M, Calvert S. Brief report: vocabulary acquisition for children with autism: teacher or computer instruction. J Autism Dev Disord 2000; 30 (04) 359-362
  • 29 Parsons S, Mitchell P, Leonard A. The use and understanding of virtual environments by adolescents with autistic spectrum disorders. J Autism Dev Disord 2004; 34 (04) 449-466
  • 30 Pennington RC. Computer-assisted instruction for teaching academic skills to students with autism spectrum disorders: a review of literature. Focus Autism Other Dev Disabl 2010; 25 (04) 239-248
  • 31 Madsen M, El Kaliouby R, Goodwin M, Picard R. , eds. Technology for just-in-time in-situ learning of facial affect for persons diagnosed with an autism spectrum disorder. Proceedings of the 10th international ACM SIGACCESS Conference on Computers and Accessibility. ACM; 2008
  • 32 Liu R, Salisbury JP, Vahabzadeh A, Sahin NT. Feasibility of an autism-focused augmented reality smartglasses system for social communication and behavioral coaching. Front Pediatr 2017; 5: 145
  • 33 Washington P, Voss C, Haber N. , et al , eds. A wearable social interaction aid for children with autism. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM; 2016
  • 34 Harms MB, Martin A, Wallace GL. Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychol Rev 2010; 20 (03) 290-322
  • 35 Maglione MA, Gans D, Das L, Timbie J, Kasari C. ; Technical Expert Panel; HRSA Autism Intervention Research – Behavioral (AIR-B) Network. Nonmedical interventions for children with ASD: recommended guidelines and further research needs. Pediatrics 2012; 130 (Suppl. 02) S169-S178
  • 36 Castelli F. Understanding emotions from standardized facial expressions in autism and normal development. Autism 2005; 9 (04) 428-449
  • 37 Ozonoff S, Pennington BF, Rogers SJ. Are there emotion perception deficits in young autistic children?. J Child Psychol Psychiatry 1990; 31 (03) 343-361
  • 38 Baron-Cohen S, Golan O, Ashwin E. Can emotion recognition be taught to children with autism spectrum conditions?. Philos Trans R Soc Lond B Biol Sci 2009; 364 (1535): 3567-3574
  • 39 Corden B, Chilvers R, Skuse D. Avoidance of emotionally arousing stimuli predicts social-perceptual impairment in Asperger's syndrome. Neuropsychologia 2008; 46 (01) 137-147
  • 40 Berggren S, Engström AC, Bölte S. Facial affect recognition in autism, ADHD and typical development. Cogn Neuropsychiatry 2016; 21 (03) 213-227
  • 41 Fridenson-Hayo S, Berggren S, Lassalle A. , et al. Basic and complex emotion recognition in children with autism: cross-cultural findings. Mol Autism 2016; 7 (01) 52
  • 42 Hopkins IM, Gower MW, Perez TA. , et al. Avatar assistant: improving social skills in students with an ASD through a computer-based intervention. J Autism Dev Disord 2011; 41 (11) 1543-1555
  • 43 Kuusikko S, Haapsamo H, Jansson-Verkasalo E. , et al. Emotion recognition in children and adolescents with autism spectrum disorders. J Autism Dev Disord 2009; 39 (06) 938-945
  • 44 Russo-Ponsaran NM, Evans-Smith B, Johnson J, Russo J, McKown C. Efficacy of a facial emotion training program for children and adolescents with autism spectrum disorders. J Nonverbal Behav 2016; 40 (01) 13
  • 45 Ben-Sasson A, Hen L, Fluss R, Cermak SA, Engel-Yeger B, Gal E. A meta-analysis of sensory modulation symptoms in individuals with autism spectrum disorders. J Autism Dev Disord 2009; 39 (01) 1-11
  • 46 Hilton CL, Harper JD, Kueker RH. , et al. Sensory responsiveness as a predictor of social severity in children with high functioning autism spectrum disorders. J Autism Dev Disord 2010; 40 (08) 937-945
  • 47 Leekam SR, Nieto C, Libby SJ, Wing L, Gould J. Describing the sensory abnormalities of children and adults with autism. J Autism Dev Disord 2007; 37 (05) 894-910
  • 48 Liss M, Saulnier C, Fein D, Kinsbourne M. Sensory and attention abnormalities in autistic spectrum disorders. Autism 2006; 10 (02) 155-172
  • 49 Tonacci A, Billeci L, Tartarisco G. , et al. [Formula: see text]Olfaction in autism spectrum disorders: a systematic review. Child Neuropsychol 2017; 23 (01) 1-25
  • 50 Allen CW, Silove N, Williams K, Hutchins P. Validity of the social communication questionnaire in assessing risk of autism in preschool children with developmental problems. J Autism Dev Disord 2007; 37 (07) 1272-1278
  • 51 Association WM. World Medical Association Declaration of Helsinki: recommendations guiding physicians in biomedical research involving human subjects [Internet]. Somerset West, RSA: 48th World Medical Association General Assembly; 1996 Oct. [cited 2010 Jun 8] Nord Med 1992; 107: 24-25
  • 52 Roid GH. Stanford-Binet Intelligence Scales. Itasca, IL: Riverside Publishing; 2003
  • 53 Constantino JN, Gruber CP. Social Responsiveness Scale. 2nd ed (SRS-2). Los Angeles, CA: Western Psychological Services; 2012
  • 54 LoBue V, Thrasher C. The Child Affective Facial Expression (CAFE) set: validity and reliability from untrained adults. Front Psychol 2015; 5: 1532
  • 55 Ekman P, Friesen WV, O'Sullivan M. , et al. Universals and cultural differences in the judgments of facial expressions of emotion. J Pers Soc Psychol 1987; 53 (04) 712-717
  • 56 Dichter GS, Felder JN, Green SR, Rittenberg AM, Sasson NJ, Bodfish JW. Reward circuitry function in autism spectrum disorders. Soc Cogn Affect Neurosci 2012; 7 (02) 160-172
  • 57 DePape A-MR, Hall GB, Tillmann B, Trainor LJ. Auditory processing in high-functioning adolescents with Autism Spectrum Disorder. PLoS One 2012; 7 (09) e44084
  • 58 Smith EG, Bennetto L. Audiovisual speech integration and lipreading in autism. J Child Psychol Psychiatry 2007; 48 (08) 813-821
  • 59 Murphy KP. Machine Learning: A Probabilistic Perspective. Cambridge, MA: MIT Press; 2012
  • 60 Nichols TE, Holmes AP. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum Brain Mapp 2002; 15 (01) 1-25
  • 61 Constantino JN, Charman T. Gender bias, female resilience, and the sex ratio in autism. J Am Acad Child Adolesc Psychiatry 2012; 51 (08) 756-758
  • 62 Dworzynski K, Ronald A, Bolton P, Happé F. How different are girls and boys above and below the diagnostic threshold for autism spectrum disorders?. J Am Acad Child Adolesc Psychiatry 2012; 51 (08) 788-797
  • 63 Howard MA, Cowell PE, Boucher J. , et al. Convergent neuroanatomical and behavioural evidence of an amygdala hypothesis of autism. Neuroreport 2000; 11 (13) 2931-2935
  • 64 Laski KE, Charlop MH, Schreibman L. Training parents to use the natural language paradigm to increase their autistic children's speech. J Appl Behav Anal 1988; 21 (04) 391-400