A Mobile Game Platform for Improving Social Communication in Children with Autism: A Feasibility Study

Abstract

Background Many children with autism cannot receive timely in-person diagnosis and therapy, especially in situations where access is limited by geography, socioeconomics, or global health concerns such as the current COVD-19 pandemic. Mobile solutions that work outside of traditional clinical environments can safeguard against gaps in access to quality care.

Objective The aim of the study is to examine the engagement level and therapeutic feasibility of a mobile game platform for children with autism.

Methods We designed a mobile application, GuessWhat, which, in its current form, delivers game-based therapy to children aged 3 to 12 in home settings through a smartphone. The phone, held by a caregiver on their forehead, displays one of a range of appropriate and therapeutically relevant prompts (e.g., a surprised face) that the child must recognize and mimic sufficiently to allow the caregiver to guess what is being imitated and proceed to the next prompt. Each game runs for 90 seconds to create a robust social exchange between the child and the caregiver.

Results We examined the therapeutic feasibility of GuessWhat in 72 children (75% male, average age 8 years 2 months) with autism who were asked to play the game for three 90-second sessions per day, 3 days per week, for a total of 4 weeks. The group showed significant improvements in Social Responsiveness Score-2 (SRS-2) total (3.97, p <0.001) and Vineland Adaptive Behavior Scales-II (VABS-II) socialization standard (5.27, p = 0.002) scores.

Conclusion The results support that the GuessWhat mobile game is a viable approach for efficacious treatment of autism and further support the possibility that the game can be used in natural settings to increase access to treatment when barriers to care exist.

Keywords

digital therapeutics - mobile game - mobile health - AI-medicine - autism spectrum disorder - developmental disorders - behavioral therapy - clinical testing - smartphone - patient population - mental health

Background and Significance

Autism spectrum disorder (ASD) affects at least 214 million children worldwide, including 1 million children at or under the age of 10 in the United States.[1] [2] Autism's prevalence in the United States rose from 1 in 149 children in 2000 to 1 in 59 in 2014 (a 2.5× increase), with most recent estimates projecting a further increase.[1] [3] Children with ASD exhibit restricted and repetitive behaviors, communication, and social skill deficits including difficulties with emotion, recognition, and joint attention.[4] [5] [6] [7] [8] While multiple studies have demonstrated that the delivery of behavioral therapy as early as possible can effectively treat and even eliminate an ASD diagnosis,[9] [10] [11] [12] [13] wait times to receive a diagnostic evaluation can,[14] exceed 14 months,[15] and out-of-pocket expenses can reach $80,000 per year[16] leaving one in three autistic children in the United States unable to receive standard care. Geographic isolation and coronavirus disease 2019 (COVID-19) related lockdowns present further limitations in access to consistent and adequate treatment.[17] [18]

Mobile forms of diagnosis and therapy that can function at home offer a way to affordably meet the demand for early intervention services for children with ASD and related neurodevelopmental disorders. We previously demonstrated that wearable augmented reality glasses such as Superpower Glass,[19] [20] [21] [22] [23] [24] [25] can deliver significant therapeutic benefits to children with autism using a fun exchange between the autistic child and a caregiver through games such as “Capture The Smile” where the child must get the caregiver to express a series of emotions.[26] Other mobile interventions, such as AKL-T01 (EndeavorRx, Akili Interactive Labs, San Francisco, California , United States), the first game-based therapeutic to receive market clearance by the FDA in June 2020,[27] have shown therapeutic effectiveness in attention deficit/hyperactivity disorder (ADHD),[28] anxiety,[29] depression,[30] and other mental health conditions.[31] This growing body of evidence highlights the potential for scalable digital therapies to provide access to care to patient populations in the United States and globally.

Developing engaging and effective mobile game-based digital therapeutics requires several sequential steps from ideation to clinical testing and validation. The first step beyond initial concept and storyboarding is co-design with the stakeholders to ensure that players will be sufficiently engaged in the game and to begin to circumscribe the demographics and clinical characteristics of the users for whom the digital therapy can have its highest impact. The Superpower Glass program started with co-design experiments which examined how children from different demographics and a wide range of autism severity levels interacted with our device.[24] [32] Likewise, the clinical development of AKL-T01 began with a feasibility study of the effectiveness of a prototype of the intervention in a wide range of children with sensory processing dysfunction with and without ADHD.[33] Following this exploratory stage, to settle on the core functionalities of the digital intervention, researchers must pilot test the intervention's utility and potential in a treatment-only clinical feasibility test. A feasibility test of the wearable augmented reality Superpower Glass system showed significant changes in meaningful clinical end points in 14 autistic children of an average age of 7 years 6 months (SD =  2.51 years).[34] Similarly, AKL-T01 was tested in a study which allowed the researchers to decide on a final clinical target and a set of product characteristics that showed the greatest potential for therapeutic benefit among their target patients.[35] A key next step, often a final step to commercial pathways, is to test the effect of the digital therapeutic in a formal randomized controlled trial that follows an intention-to-treat trial protocol. Both Superpower Glass[26] and AKL-T01[28] were tested in randomized controlled trials and showed significant positive changes on their primary endpoint measures in the treatment group.

We have followed our digital therapeutic translational framework outlined above to test the feasibility of GuessWhat's computer–human interaction design, user experience and engagement in laboratory[36] and home environments[37] [38] [39] and to optimize the system to be both fun and therapeutic. Specifically, we prototyped the game with eight families over 6 weeks with a combination of in-person focus group sessions, beta testing, and at-home game play. We then launched the application on the Android and iOS application stores for free download and use and marketed the game's availability through Facebook and other social media outlets to encourage players (irrespective of autism diagnosis or any other characteristic) to try the game. We gathered play metrics and electronic feedback from this virtual collection of players to further refine the game design. We evaluated the data feed to settle on the 90-second game session timeframe for the charades challenges as optimal both for engagement and for data acquisition. These aspects of the human computer interaction design led to the version tested here, which was optimized for engagement and treatment potential. In the present study, we test this version of the game's feasibility as an at-home digital therapeutic for children with autism between the ages of 3 and 12 years (the current ages where we have performed user-testing and engagement design iterations). This study is an important and necessary step toward a formal randomized controlled trial.

Methods

Study Design and Participants

This study was conducted remotely. Participants were recruited through GuessWhat's existing userbase, The Hartwell Foundation's KidsFirst autism research database, ResearchMatch.org, and Facebook advertisements. All participating families met the following criteria: (1) able to read and speak English, (2) possessed or had access to a compatible iOS or Android device with internet access, (3) the parent was 18+ years old, (4) the child was between 3 and 12 years of age and diagnosed with ASD. To safeguard against the potential for self-reporting bias, we required the caregiver to confirm that their child's autism diagnosis came from a formal medical assessment. We asked the caregiver to choose a diagnostic label from a menu of choices including ASD, autistic disorder, pervasive developmental disorder-not otherwise specified, Asperger syndrome, ADHD/ADD, anxiety, speech and language delay. In addition, we required participants to report on the specific type(s) of therapy being administered to their child. We also asked participants to fill out fields associated with the Mobile Autism Risk Assessment,[40] a machine learning tool validated to have high accuracy for autism screening. Finally, our clinical team reviewed all videos shared by participants for exhibition by the autistic child of symptoms and features consistent with an autism diagnosis. All participants provided informed consent online on behalf of their child prior to participation. Ethical approval was obtained from Stanford University's Institutional Review Board prior to all research activities. This approval included a Data Risk Assessment performed by the University Privacy Office that found our system to be compliant with Stanford University, State of California, and Federal privacy regulations.

Mobile Game Platform

GuessWhat is a mobile application available for at-home use through any iOS and Android device ([Fig. 1A]). The game uses a form of mobile charades to create opportunities for social exchange between a child and a playing partner centered around face and eye contact and expanded range of emotion recognition and expressivity for the child. A caregiver holds a smartphone on their forehead while prompts appear on the screen. The child must act out the prompts so that the caregiver will be able to guess what the child is acting ([Fig. 1B, C]). The gameplay integrates key components of applied behavior analysis (ABA),[41] discrete trial training,[42] and pivotal response training[43] ([Table 1]) to engage children in self-initiated and naturalistic imitation[44] and sociodramatic and symbolic play[45] with their social partner (e.g., their parent). In-app daily, weekly, and monthly challenges and reward mechanisms (including audio affirmation, coins, and the unlocking of new game decks and achievement badges) reinforce positive progression through the game and maintain the child's interest through time ([Fig. 1], day f).

Fig. 1 The GuessWhat Mobile User Experience. (A) GuessWhat is a mobile game available for any smartphone device. In a typical game session, (B) the parent holds the smartphone to their forehead and tries to guess what the child is acting in response to the prompt shown on the phone's screen. Upon guessing, the parent tilts the phone to proceed to the next prompt through the end of the 90-second session. (C) After each 90-second game, parent and child can review together, enhancing and strengthening the learning. In-app (D) game modes, (E) unlocking deck and character choices based on coins earned, and (F) activity-based achievement badges reinforce positive progression and ensure optimal child engagement through time.

Table 1 GuessWhat game elements and treatment focus areas. The parent holds the phone on forehead, variable image prompts appear on screen, and child acts what is displayed ([Fig. 1]). If parent can guess the prompt correctly, they tilt the phone downward to earn a coin and move to the next prompt. If unable to guess, the parent tilts the phone upward to skip to the next prompt. The game in its current form comes with over 40 different deck choices that can be used alone or mixed into sessions lasting 90 seconds each. Earned coins enable unlocking of new decks and the purchase of new game characters desired by the child. The mobile application is available on Android and iOS; both versions are linked from guesswhat.stanford.edu

GuessWhat game element	Treatment focus area
Charades: This is a primary game mode. Act out images displayed on screen. Vary the acting challenges and try to score higher with each session.	Eye contact, joint attention, theory of mind, imitation, expressive and receptive communication, nonverbal social awareness, conceptual (abstract) reasoning; flexible thinking.
Selecting and mixing decks: Choose from 40 different decks that feature different animals, sports, occupations, environments, and more.	Self-initiation, independence, comprehension of and response to multiple cues, reinforcement of variable interests/behaviors.
Earning coins: Correct guesses by play partner rewarded with coins that allow purchase and unlocking of new game decks and game characters.	Social motivation
Turn taking mode: Child and parent prompted to control the phone and take turns acting out the challenges displayed on the phone screen.	Turn taking, sharing, social initiations
Video playback: Watch yourself perform the challenge.	Reinforcement learning and course correction
Motion challenges: Sports, gestures, objects, chores decks.	Gross and fine motor skills
Sound challenges: sing-along, pledge allegiance, poetry recitals.	Speech and language
Emotion challenges: emojis, human faces decks, cartoon character faces.	Facial emotion recognition and expression

Each play session begins by the parent selecting one or more of several prompt decks, such as Faces, Dances, or Emojis, which aim at core behavioral targets such as receptive and expressive communication, joint attention, multiple cue response, gross and fine motor skills ([Table 1]).[9] [41] [42] [43] Once a game deck(s) has been selected, the parent presses “Play” and holds the phone to their forehead with the screen facing the child. The child then interprets the prompt displayed on the screen and acts out what they see (e.g., “surprise” for the Faces deck or “the floss” for the Dances deck). The parent must then guess what the child is acting out and confirm if they are correct by communicating via verbal or social cues and tilting the phone up or down. GuessWhat uses the phone's accelerometer to detect a tilt in the forward direction indicating a correct guess or a tilt back indicating “skip” to proceed forward to the next prompt. After 90 seconds, the game session is complete, and the results are presented back to the user as how many prompts were guessed correctly versus skipped. The parent is then given the opportunity to review the answers and the gameplay video with the child, and to save the video to their phone, share it with our laboratory, or delete it altogether. When a user chooses to share their gameplay video, it is stored in a Health Insurance Portability and Accountability Act (HIPAA) compliant encrypted cloud database for further research.

Participant Procedures

All participants were asked to download and start playing GuessWhat at the beginning of the study. Families were instructed to play the game for at least three 90-second sessions per day, 3 days per week, for 4 weeks, for a total of 54 minutes of gameplay over 1 month. Participants were directed to play any of the available game decks, with particular emphasis on Faces and Emojis, which were specifically designed to encourage emotion recognition and social communication. Parent-reported surveys were collected for all participants immediately before and after the 4-week testing period. Participants were sent an Amazon gift card for the completion of study procedures.

Measures

Gameplay metrics and qualitative feedback were collected from all participants who used the GuessWhat application. Gameplay metrics were collected throughout the 4-week intervention period and included game deck selection, number of 90-second game sessions initiated and completed, and total play time per session, per login, and for the entire study period.

A qualitative feedback measure created by our research team assessed the GuessWhat players' user experience at the end of the 4-week period. This measure includes 11 free-text and multiple-choice items capturing participants' feedback and suggestions, technical difficulties they experienced while playing the app, preferred game modes, and likelihood of using the app beyond the study.

All study participants responded to two standardized clinical outcome measures for autism, the Social Responsiveness Scale, 2^nd edition[46] (SRS-2) and the Vineland Adaptive Behavior Scales, 2^nd edition[47] (VABS-II). Both surveys were collected at the start and end of each family's 1-month testing window.

SRS-2 is a 65-item survey intended to identify the presence and severity of social impairment in children across five social domains. Standardized scores at or above 60 are considered indicative of social impairment.

VABS-II is a 502-item survey which measures children's adaptive functioning on four main domains: communication, daily living skills, socialization, and motor skills. We collected the Socialization Domain (99 questions that measure interpersonal relationships, play and leisure time, and coping skills) and the Receptive and Expressive Communication subscales (74 items) of the Communication Domain using the Comprehensive Parent/Caregiver Form of the survey. Higher scores indicate greater social functioning.

Analysis

Usage metrics were analyzed across each participant from first date of gameplay following pre-test survey data collection through last gameplay session prior to post-test survey data collection. Free-text responses in the qualitative feedback measure recorded during post-test survey data collection were evaluated using content analysis to identify the most common technical difficulties as well as overall satisfaction and engagement with the GuessWhat mobile game. Analysis of the qualitative feedback measure also included the quantification of multiple-choice radio select options measuring how many participants intended to continue playing the app following study completion.

Changes in SRS-2 and VABS-II scores were assessed using means comparisons with two-tailed paired sample t-tests. Values 1.5 times above or below the interquartile range were excluded from the analysis. Families who did not record GuessWhat play attempts but completed all other procedures were analyzed as a separate group for comparison.

Results

We tested the therapeutic feasibility of our mobile game with 72 autistic children (75% male, average age 8 years 2 months, minimum age 3 years 10 months, maximum age 12 years 6 months) ([Fig. 2]). Sixty-two (86%) players were receiving standard autism therapy treatments, with 60 (83%) receiving two types or more during the study ([Table 2]). A comparison control group that did not play the game but completed all other procedures included 19 children (84% male, average age 8 years 5 months, minimum age 3 years 11 months, maximum 12 years 4 months). Eighteen (95%) children in the comparison group received some form of autism-related therapy, with 13 (68%) receiving two types or more ([Table 2], [Supplementary Fig. S1], available in the online version). Prior to the start of our GuessWhat treatment, we did not find significant differences between the two groups in the two baseline measures, the Social Responsiveness Scale (SRS-2) and the Vineland Adaptive Behavior Socialization Standard Score ([Table 2]). We did not find any disagreements between our independent confirmation of the autism diagnosis and the self-reported autism diagnosis in either the treated or the comparison group.

Fig. 2 Study groups. Ninety-one families were assessed for eligibility and completed pre- and post-test surveys between March 2019 and December 2020. Seventy-two families played GuessWhat as instructed. Nineteen families who did not record GuessWhat play attempts but fulfilled all other requirements were analyzed as a separate group for comparison.

Table 2 User demographics and survey scores for the 72 autism families in the GuessWhat treatment group and the 19 matched participants in the comparison control group (autism families who did not play GuessWhat but completed all other procedures)

Demographic information, Mean (SD)/Percent (N)iii		All users (N = 72)	Users with ≥28 sessions (N = 17)	Users with 4 wk of consistent usage (N = 13)	Comparison group (N = 19)
Age (years)		8.17 (2.25)	7.97 (2.58)	7.9 (2.13)	8.4 (2.89)
Gender (% male)		75% (54)	80% (16)	85% (11)	84% (16)
Comorbidity	Attention deficit disorder	14% (10)	15% (3)	8% (1)	21% (4)
	Anxiety disorder	6% (4)	0% (0)	0% (0)	21% (4)
	Bipolar disorder	1% (1)	0% (0)	8% (1)	0% (0)
	Sensory processing disorder	4% (3)	0% (0)	0% (0)	5% (1)
Race and ethnicity	Caucasian/Euro-American	72% (52)	65% (13)	77% (10)	68% (13)
	African American	7% (5)	10% (2)	0% (0)	26% (5)
	East Asian/Asian American	10% (7)	10% (2)	8% (1)	0% (0)
	South Asian/Indian American	4% (3)	10% (2)	0% (0)	5% (1)
	Middle Eastern/Arab American	1% (1)	0% (0)	0% (0)	0% (0)
	Pacific Islander	0% (0)	0% (0)	0% (0)	11% (2)
	Native American/Alaskan Native	1% (1)	0% (0)	0% (0)	5% (1)
	Hispanic/Latino/Spanish Origin	22% (16)	20% (4)	8% (1)	21% (4)
	Unknown/Not listed	6% (4)	5% (1)	77% (10)	0% (0)
Concurrent therapy enrollment	Applied behavior analysis (ABA)	47% (34)	50% (10)	62% (8)	58% (11)
	Social skills therapy	35% (25)	40% (8)	31% (4)	37% (7)
	Special education classes	44% (32)	30% (6)	38% (5)	53% (10)
	Speech-language pathology	74% (53)	70% (14)	69% (9)	47% (9)
	Occupational therapy	58% (42)	65% (13)	62% (8)	68% (13)
	No therapy	14% (10)	15% (3)	8% (1)	5% (1)
	1 type of therapy only	2.8% (2)	5% (1)	0% (0)	26% (5)
	2+ types of therapy	83% (60)	80% (16)	0% (0)	68% (13)
Clinical evaluations
Social Responsiveness Scale (SRS-2), intake score	Social awareness	75.24 (9.85)	74.29 (9.42)	76.54 (6.71)	79.71 (10.44)
	Social cognition	77.46 (8.36)	76.47 (9.07)	77.77 (6.1)	76.22 (11.63)
	Social communication	78.5 (9.51)	78.53 (11.23)	80.08 (4.05)	79.24 (9.29)
	Social motivation	69.67 (11.14)	68.12 (11.94)	67.62 (8.71)	70.4 (5.74)
	Restricted and repetitive behavior	79.04 (9.77)	78.24 (9.97)	82.31 (7.03)	77 (11.06)
	Total*	80 (8.43)	78.82 (10.56)	81 (4.56)	80.82 (8.18)
Vineland adaptive behavior scales-II (VABS-II), intake score	Receptive communication	9.55 (3.31)	9.76 (3.88)	8.85 (1.63)	8.93 (2.43)
	Expressive communication	9.86 (4.04)	10.94 (5.02)	9 (1.28)	8.22 (4.05)
	Socialization relationships	8.99 (3.69)	9.56 (3.79)	8.54 (2.02)	7.78 (3.25)
	Socialization play	8.55 (3.18)	10.18 (5.87)	8.77 (2.3)	7.67 (4.56)
	Socialization adaptive	10.82 (3.21)	12 (3.82)	10.23 (2.13)	9.65 (3.14)
	Socialization standard Total*	68.94 (15.53)	73.44 (20.85)	68.08 (6.75)	65.17 (20.52)

Note: The baseline total scores (designated with ^a) for the two primary measures did not differ significantly between the treatment and control groups, regardless of usage patterns.

All players reported high engagement and satisfaction with the GuessWhat mobile application. Sixty-seven percent of families expressed intention to continue playing beyond the study period. Eleven families reported experiencing technical difficulties, five of whom were able to resolve the issues during the testing period. The most common issues experienced by these users were trouble logging in (n = 5), poor internet connection (n = 3), or the application freezing or crashing (n = 2). Thirteen families reported difficulties such as confusion about how to play the game (n = 6) or prompts that were too difficult (n = 4). Seven families expressed difficulty interpreting the game instructions.

GuessWhat users played an average of 17.68 (SD = 18.04) sessions over 5.14 (SD = 4.63) days during the 4-week testing period, 49% of the recommended use. Participants selected the Emojis deck in 16% of the game sessions, Faces in 12%, Animals in 13%, followed by Objects (8%), Jobs (7%), Sports (7%), and all others (37%) ([Fig. 3]). Proper gameplay was verified through videos recorded during game play and that were shared with our team ([Table 3]).

Fig. 3 GuessWhat game decks. Users played a total of 1,965 game decks over the course of the study. The Emojis deck was chosen in 16% of all initiated game sessions, followed by Animals (13%), Faces (12%), Objects (8%), Jobs (7%), and Sports (7%). Cumulatively, all other decks were chosen in 37% of game sessions.

Table 3 GuessWhat usage statistics. Gameplay statistics were recorded over 1 mo for all 72 players in the treatment group

	Game start		Game end		Game share		Points
Game	Num.	Avg.	Num.	Avg.	Num.	Avg.	Num.	Avg.
Emojis	316 (16%)	4.33 (SD = 5.98)	251 (15%)	3.44 (SD = 4.68)	208 (14%)	2.85 (SD = 4.1)	2,220 (25%)	30.41 (SD = 50.44)
Faces	233 (12%)	3.19 (SD = 4.34)	201 (12%)	2.75 (SD = 4.11)	172 (12%)	2.36 (SD = 3.82)	1,950 (22%)	26.71 (SD = 50.51)
Animals	250 (13%)	3.42 (SD = 7.15)	223 (13%)	3.05 (SD = 6.57)	202 (14%)	2.77 (SD = 6.55)	1,818 (21%)	24.90 (SD = 52.49)
Objects	155 (8%)	2.12 (SD = 4.09)	136 (8%)	1.86 (SD = 3.52)	111 (8%)	1.52 (SD = 3.34)	884 (10%)	12.11 (SD = 27.52)
Jobs	136 (7%)	1.86 (SD = 5.05)	124 (7%)	1.70 (SD = 4.54)	103 (7%)	1.41 (SD = 4.32)	720 (8%)	9.86 (SD = 27.37)
Sports	134 (7%)	1.84 (SD = 5.07)	113 (7%)	1.55 (SD = 4.58)	93 (6%)	1.27 (SD = 4.43)	746 (8%)	10.22 (SD = 25.88)
Other	740 (38%)	10.15 (SD = 24.81)	665(39%)	9.11 (SD = 21.95)	591 (40%)	8.10 (SD = 21.55)	488 (6%)	6.68 (SD = 95.66)
Тotal decks	1,964	26.92 (SD = 47.08)	1713	23.47 (SD = 42.46)	1,480	20.27 (SD = 41.32)	8,826	120.90 (SD = 158.86)
Total sessions	1,291	17.68 (SD = 18.04)	1105	15.14 (SD = 15.81)	959	13.14 (SD = 15.32)		183.93 (N/A)

Users demonstrated an average of 3.97-point improvement in SRS-2 total T-score (SD = 4.92, p < 0.001) as well as an improvement of 5.27 points on the VABS-II socialization standard score (SD = 9.29, p = 0.002). The game players who played more than 28 sessions (n = 17) showed a 3.82-point (SD = 5.70, p = 0.01) positive change on the SRS-2 Total T-score and a 6.21-point (SD = 8.13, p = 0.11) improvement on the VABS-II Socialization Standard score. The game players who played consistently over all 4 weeks (n = 13) showed a 2.85-point (SD = 4.56, p = 0.03) positive change on the SRS-2 total T-score and 5.46-point (SD = 6.75, p = 0.58) improvement on the VABS-II socialization standard score. The 19 families who did not play GuessWhat but completed all other procedures showed no significant change in scores recorded by either survey. Results of the primary means comparisons are presented in [Table 4] (subsection scores available in [Supplementary Table S1], available in the online version).

Table 4 Change in parent-reported survey scores by user group. Results include 72 families who played GuessWhat and a comparison group of 19 matched participants who did not play GuessWhat but completed all other procedures. Reductions in SRS-2 and increases in VABS-II scores are indicative of improvement and vice versa

Measure	Mean change from pre-test to post-test (SD)
	All treated players	p-Value	Users treated with ≥28 sessions	p-Value	Users with 4 wk. treatment consistency	p-Value	Comparison group	p-Value
SRS-2 total T-score	−3.97 (SD = 4.92)	<0.001[c]	−3.82 (SD = 5.70)	0.01[a]	−2.85 (SD = 4.56)	0.03[a]	−4.71 (SD = 7.92)	0.10
VABS-II socialization standard score	5.27 (SD = 9.29)	0.002[b]	6.21 (SD = 8.13)	0.11	5.46 (SD = 6.75)	0.58	−1.61 (SD = 12.37)	0.62

^a p <0.05.

^b p <0.01.

^c p <0.001 between cohorts from a two-tailed paired sample t-test.

Discussion

Our study examined the potential for the mobile game GuessWhat to provide a therapeutic effect for autistic children and families who play the game for a period of 1 month. The game fosters a prosocial exchange between the child and playing partner in short 90-second sessions designed for social skills development including eye contact and emotion recognition. Out of the 91 families assessed for eligibility who completed pre- and post-test surveys, 72 (79%) successfully played one or more GuessWhat game sessions. The 19 (21%) families who did not play the game at all were assigned to a separate comparison group. The 72 GuessWhat users exhibited significant improvements in SRS-2 and VABS-II scores, while the untreated comparison group showed no significant differences in either measure over the same 4-week timeframe.

Average improvements in SRS-2 total scores were in line with minimal clinically important differences (MCIDs) required for a shift from severe to moderate, moderate to mild, or mild autism severity to within normal limits.[48] Average improvements in VABS-II Socialization domain scores exceeded the MCID of 3.7 points.[49] Interestingly, these therapeutic effects were evident even with adherence to dose recommendations at approximately 50% of the recommended game play frequency. GuessWhat users averaged 5.14 gameplay days over the 4-week testing period even though they were asked to play a minimum of 12 days (three sessions, 3 days a week over 4 weeks). Potential reasons for the lower adherence include technical difficulties experienced by some families, decreased child engagement with the app over time due to boredom, and/or decreased motivation by the parent to continue using the intervention. While these lower usage patterns demonstrate the need for an enhanced user interface including reminder and reward systems, the observed gains in socialization provide support for the digital game's potential as a useful form of autism therapy. The results also support previous game therapy research showing that parental involvement, a major design focus for our system, positively impacts the therapeutic effect.[50]

Players who played more than 28 GuessWhat sessions and/or played during each week of the intervention period showed larger improvements on VABS-II Socialization Standard scores and on VABS-II Receptive and Expressive Communication scores suggesting that improvements may be greater with higher compliance to the recommended gameplay ([Table 4]). This dose-dependency in therapeutic gains also supports the likelihood that the observed effects were likely not due to a placebo effect. Further tests are needed to confirm the therapeutic effectiveness, to determine optimal levels of gameplay, to design the most effective means of ensuring adherence to it, and to show the potential for sustained gains after therapy periods conclude.

Limitations

This research demonstrates how a new type of mobile game-based therapy may be used from the convenience of people's homes for improving social acuity in children with autism. One limitation of the study was the use of nonblinded, parent-administered intervention and outcome measures, which risks biased reporting. It can be expected that potential biases in parent reporting would manifest as stronger on the SRS-2 scale than on VABS-II, since the latter asks more quantitative questions and therefore leaves less opportunity for subjective judgment. Accordingly, the lack of discrepancies between SRS-2 and VABS-II results for GuessWhat players ([Table 4], [Supplementary Table S1], available in the online version) demonstrates that the socialization gains observed following a month of gameplay were not primarily attributable to biases in parental reporting. Instead, the game's therapeutic effect is likely due to the increased frequency of (1) structured social engagement between parent and child, and (2) practice of facial and emotion recognition skills during the charades-style interaction.

A second limitation of the present study was potential self-selection bias. It may be the case that the 19 families who did not attempt to play GuessWhat but completed all other study procedures had lower motivation to participate in the study. A lack of motivation in this comparison group could have therefore resulted in reduced gains measured by either SRS-2 or VABS-II parent-reported surveys. Nevertheless, the inconsistent changes between the two outcome measures (SRS-2 scores improved while VABS-II worsened) and the large standard deviation observed in the comparison group ([Table 4], [Supplementary Table S1] [available in the online version]) indicate that reduced gains due to lack of motivation did not sufficiently account for the differences observed between GuessWhat players and non-players. A larger comparison group will be needed to confirm these results.

A third limitation was that the demographics of our study sample were predominantly male and Caucasian ([Table 2]). Recruitment of balanced populations is challenged by the 4:1 male:female prevalence of autism[1] and underdiagnosis and undertreatment among non-Caucasian autism families.[2] Simple correlations between participant demographics and usage data or outcome measures yielded no clear conclusions about the best responders. We will use GuessWhat's advantages as a digital intervention to develop new game decks that will appeal to participants of any gender or ethnicity and to recruit a more balanced cohort among a wider range of participants from all over the United States and the world in future studies.

Finally, the enrollment of 32 GuessWhat treatment and 18 of the 19 control participants during COVID-19-related school closures may have introduced additional confounding factors such as loss of regular therapy, childcare and school services, and others. Changes in therapy and lifestyle during the intervention period or at a post-test remain a topic for future investigation.

Conclusion

This study demonstrates that GuessWhat can improve socialization in children with autism in an engaging and accessible manner. The results support the potential of game-based mobile systems to augment the standard approaches to autism therapy, ideally increasing the continuity of behavioral therapy and preventing loss in services where access to care is limited.

Clinical Relevance Statement

The results of this study indicate that game-based digital therapeutics may be an effective to deliver behavioral therapy for children with autism spectrum disorder. This outcome is particularly relevant in the context of the current coronavirus disease pandemic and restrictions to in-person clinical services. Mobile tools such as GuessWhat present a viable way for bridging gaps in continuity of care and ensuring that all families can access effective and engaging behavioral therapy from the comfort of their home.

Multiple Choice Questions

1. What is the first step (after prototyping) in the design of a novel digital therapeutic?

   • Animal testing.

   • Computer modeling.

   • Co-design experimentation.

   • Phase I clinical trial.

   Correct Answer: The correct answer is option c. The design of novel digital therapeutic begins with a series of co-design experiments done with the target patient population intended to optimize engagement, to begin to understand the dose–response relationship, to refine hypotheses regarding the expected clinical treatment effect, and to generally understand the ways in which the stakeholders will interact with a prototype of the product.

2. When did FDA begin clearing game-based digital therapeutics for market use?

   • 1990s.

   • 2000s.

   • 2010s.

   • 2020s.

   Correct Answer: The correct answer is option d. The Food and Drug Administration cleared the first game-based digital intervention for market use in the United States in June 2020.

Conflict of Interest

D.P.W. is the founder of Cognoa. This company is developing digital health solutions for pediatric care. A.K. works as a part-time consultant to Cognoa. All other authors declare no conflict of interest.

Acknowledgments

The authors would like to thank all of the families who participated in this research study for their time and contributions. We also acknowledge generous support from David Orr, Imma Calvo, Bobby Dekesyer, and Peter Sullivan.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was reviewed by Stanford University's Institutional Review Board.

• References

• 1 Boyle CA, Boulet S, Schieve LA. et al. Trends in the prevalence of developmental disabilities in US children, 1997-2008. Pediatrics 2011; 127 (06) 1034-1042
• 2 Baio J, Wiggins L, Christensen DL. et al. Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR Surveill Summ 2018; 67 (06) 1-23
• 3 Nicholas JS, Charles JM, Carpenter LA, King LB, Jenner W, Spratt EG. Prevalence and characteristics of children with autism-spectrum disorders. Ann Epidemiol 2008; 18 (02) 130-136
• 4 American Psychiatric Association. Diagnostic and statistical manual of mental disorders (5th ed.), 2013.. Available at: https://doi.org/10.1176/appi.books.9780890425596
• 5 Kasari C, Freeman S, Paparella T. Joint attention and symbolic play in young children with autism: a randomized controlled intervention study. J Child Psychol Psychiatry 2006; 47 (06) 611-620
• 6 Charpentier J, Kovarski K, Houy-Durand E. et al. Emotional prosodic change detection in autism spectrum disorder: an electrophysiological investigation in children and adults. J Neurodev Disord 2018; 10 (01) 28
• 7 Dawson G, Webb SJ, McPartland J. Understanding the nature of face processing impairment in autism: insights from behavioral and electrophysiological studies. Dev Neuropsychol 2005; 27 (03) 403-424
• 8 Harms MB, Martin A, Wallace GL. Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychol Rev 2010; 20 (03) 290-322
• 9 Lovaas OI. Behavioral treatment and normal educational and intellectual functioning in young autistic children. J Consult Clin Psychol 1987; 55 (01) 3-9
• 10 Dawson G, Jones EJ, Merkle K. et al. Early behavioral intervention is associated with normalized brain activity in young children with autism. J Am Acad Child Adolesc Psychiatry 2012; 51 (11) 1150-1159
• 11 Sandbank M, Bottema-Beutel K, Crowley S. et al. Project AIM: autism intervention meta-analysis for studies of young children. Psychol Bull 2020; 146 (01) 1-29
• 12 Fuller EA, Kaiser AP. The effects of early intervention on social communication outcomes for children with autism spectrum disorder: a meta-analysis. J Autism Dev Disord 2020; 50 (05) 1683-1700
• 13 Landa RJ. Efficacy of early interventions for infants and young children with, and at risk for, autism spectrum disorders. Int Rev Psychiatry 2018; 30 (01) 25-39
• 14 Zwaigenbaum L, Penner M. Autism spectrum disorder: advances in diagnosis and evaluation. BMJ 2018; 361: k1674
• 15 Gordon-Lipkin E, Foster J, Peacock G. Whittling down the wait time: exploring models to minimize the delay from initial concern to diagnosis and treatment of autism spectrum disorder. Pediatr Clin North Am 2016; 63 (05) 851-859
• 16 Cidav Z, Munson J, Estes A, Dawson G, Rogers S, Mandell D. Cost offset associated with early start Denver model for children with autism. J Am Acad Child Adolesc Psychiatry 2017; 56 (09) 777-783
• 17 Lee J. Mental health effects of school closures during COVID-19. Lancet Child Adolesc Health 2020; 4 (06) 421
• 18 Ning M, Daniels J, Schwartz J. et al. Identification and quantification of gaps in access to autism resources in the United States: an infodemiological study. J Med Internet Res 2019; 21 (07) e13094-e13094
• 19 Haber N, Voss C, Wall D. Making emotions transparent: Google Glass helps autistic kids understand facial expressions through augmented-reality therapy. IEEE Spectr 2020; 57: 46-52
• 20 Kline A. et al. Superpower Glass. GetMobile. Mobile Comp Comm 2019; 23: 35-38
• 21 Voss C, Washington P, Haber N. Superpower glass: delivering unobtrusive real-time social cues in wearable systems. Paper presented at: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 1218–1226 (ACM)
• 22 Washington P, Voss C, Haber N, Tanaka S. A wearable aid for the at-home therapy of children with autism. Paper presented at: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, ACM; 2016: 2348-2354
• 23 Washington P, Voss C, Kline A. et al. SuperpowerGlass: A Wearable Aid for the At-Home Therapy of Children with Autism. Paper presented at: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 112; 2017
• 24 Daniels J, Haber N, Voss C. et al. Feasibility testing of a wearable behavioral aid for social learning in children with autism. Appl Clin Inform 2018; 9 (01) 129-140
• 25 Voss C, Haber N, Washington P. et al. Designing a holistic at-home learning aid for autism. arXiv preprint, arXiv: 200204263. 2020
• 26 Voss C, Schwartz J, Daniels J. et al. Effect of wearable digital intervention for improving socialization in children with autism spectrum disorder: a randomized clinical trial. JAMA Pediatr 2019; 173 (05) 446-454
• 27 Pena CL. CFR—Code of Federal Regulations Title 21. Silver Spring, MD: U.S. Food & Drug Administration; 2020
• 28 Kollins SH, DeLoss DJ, Cañadas E. et al. A novel digital intervention for actively reducing severity of paediatric ADHD (STARS-ADHD): a randomised controlled trial. Lancet Digit Health 2020; 2 (04) e168-e178
• 29 Chandrashekar P. Do mental health mobile apps work: evidence and recommendations for designing high-efficacy mental health mobile apps. mHealth 2018; 4: 6-6
• 30 Ly KH, Topooco N, Cederlund H. et al. Smartphone-supported versus full behavioural activation for depression: a randomised controlled trial. PLoS One 2015; 10 (05) e0126559
• 31 Lau HM, Smit JH, Fleming TM, Riper H. Serious games for mental health: are they accessible, feasible, and effective? A systematic review and meta-analysis. Front Psychiatry 2017; 7: 209-209
• 32 Daniels J, Schwartz J, Haber N, Feinstein C. 5.13 design and efficacy of a wearable device for social affective learning in children with autism. J Am Acad Child Adolesc Psychiatry 2017; 56: S257
• 33 Anguera JA, Brandes-Aitken AN, Antovich AD, Rolle CE, Desai SS, Marco EJ. A pilot study to determine the feasibility of enhancing cognitive abilities in children with sensory processing dysfunction. PLoS One 2017; 12 (04) e0172616-e0172616
• 34 Daniels J, Schwartz JN, Voss C. et al. Exploratory study examining the at-home feasibility of a wearable tool for social-affective learning in children with autism. NPJ Digit Med 2018; 1: 32
• 35 Yerys BE, Bertollo JR, Kenworthy L. et al. Brief Report: pilot study of a novel interactive digital treatment to improve cognitive control in children with autism spectrum disorder and co-occurring ADHD symptoms. J Autism Dev Disord 2019; 49 (04) 1727-1737
• 36 Kalantarian H, Washington P, Schwartz J, Daniels J, Haber N, Wall DP. Guess What?: Towards Understanding Autism from Structured Video Using Facial Affect. J Healthc Inform Res 2019; 3: 43-66
• 37 Kalantarian H, Jedoui K, Washington P, Wall DP. A mobile game for automatic emotion-labeling of images. IEEE Trans Games 2020; 12 (02) 213-218
• 38 Kalantarian H. et al. in 2018 IEEE International Conference on Healthcare Informatics. ICHI 2018. New York, NY: IEEE Computer Society; 2018: 350-352
• 39 Kalantarian H, Jedoui K, Washington P. et al. Labeling images with facial emotion and the potential for pediatric healthcare. Artif Intell Med 2019; 98: 77-86
• 40 Duda M, Daniels J, Wall DP. Clinical evaluation of a novel and mobile autism risk assessment. J Autism Dev Disord 2016; 46 (06) 1953-1961
• 41 Lovaas OI. Teaching Individuals with Developmental Delays: Basic Intervention Techniques. Pro-Ed; 2003
• 42 Smith T. Discrete trial training in the treatment of autism. Focus Autism Other Dev Disabl 2001; 16: 86-92
• 43 Koegel LK, Koegel RL, Harrower JK, Carter CM. Pivotal response intervention I: Overview of approach. J Assoc Pers Severe 1999; 24: 174-185
• 44 Ingersoll B, Schreibman L. Teaching reciprocal imitation skills to young children with autism using a naturalistic behavioral approach: effects on language, pretend play, and joint attention. J Autism Dev Disord 2006; 36 (04) 487-505
• 45 Thorp DM, Stahmer AC, Schreibman L. Effects of sociodramatic play training on children with autism. J Autism Dev Disord 1995; 25 (03) 265-282
• 46 Constantino JN, Gruber CP. Social Responsiveness Scale (SRS-2). 2nd ed.. Los Angeles, CA: Western Psychological Services; 2012
• 47 Sparrow SS, Cicchetti DV, Balla DA. Vineland Adaptive Behavior Scales, Second Edition (Vineland-II). Survey Forms Manual. Circle Pines, MN: AGS Publishing; 2005. . Available at: https://doi.org/10.1037/t15164-000
• 48 Hardan AY, Hendren RL, Aman MG. et al. Efficacy and safety of memantine in children with autism spectrum disorder: results from three phase 2 multicenter studies. Autism 2019; 23 (08) 2096-2111
• 49 Chatham CH, Taylor KI, Charman T. et al. Adaptive behavior in autism: minimal clinically important differences on the Vineland-II. Autism Res 2018; 11 (02) 270-283
• 50 Holtz BE, Murray K, Park T. Serious games for children with chronic diseases: a systematic review. Games Health J 2018; 7 (05) 291-301

Publication History

Received: 13 June 2021

Accepted: 21 September 2021

Article published online:
17 November 2021