Keywords
digital therapeutics - mobile game - mobile health - AI-medicine - autism spectrum
disorder - developmental disorders - behavioral therapy - clinical testing - smartphone
- patient population - mental health
Background and Significance
Background and Significance
Autism spectrum disorder (ASD) affects at least 214 million children worldwide, including
1 million children at or under the age of 10 in the United States.[1]
[2] Autism's prevalence in the United States rose from 1 in 149 children in 2000 to
1 in 59 in 2014 (a 2.5× increase), with most recent estimates projecting a further
increase.[1]
[3] Children with ASD exhibit restricted and repetitive behaviors, communication, and
social skill deficits including difficulties with emotion, recognition, and joint
attention.[4]
[5]
[6]
[7]
[8] While multiple studies have demonstrated that the delivery of behavioral therapy
as early as possible can effectively treat and even eliminate an ASD diagnosis,[9]
[10]
[11]
[12]
[13] wait times to receive a diagnostic evaluation can,[14] exceed 14 months,[15] and out-of-pocket expenses can reach $80,000 per year[16] leaving one in three autistic children in the United States unable to receive standard
care. Geographic isolation and coronavirus disease 2019 (COVID-19) related lockdowns
present further limitations in access to consistent and adequate treatment.[17]
[18]
Mobile forms of diagnosis and therapy that can function at home offer a way to affordably
meet the demand for early intervention services for children with ASD and related
neurodevelopmental disorders. We previously demonstrated that wearable augmented reality
glasses such as Superpower Glass,[19]
[20]
[21]
[22]
[23]
[24]
[25] can deliver significant therapeutic benefits to children with autism using a fun
exchange between the autistic child and a caregiver through games such as “Capture
The Smile” where the child must get the caregiver to express a series of emotions.[26] Other mobile interventions, such as AKL-T01 (EndeavorRx, Akili Interactive Labs,
San Francisco, California , United States), the first game-based therapeutic to receive
market clearance by the FDA in June 2020,[27] have shown therapeutic effectiveness in attention deficit/hyperactivity disorder
(ADHD),[28] anxiety,[29] depression,[30] and other mental health conditions.[31] This growing body of evidence highlights the potential for scalable digital therapies
to provide access to care to patient populations in the United States and globally.
Developing engaging and effective mobile game-based digital therapeutics requires
several sequential steps from ideation to clinical testing and validation. The first
step beyond initial concept and storyboarding is co-design with the stakeholders to
ensure that players will be sufficiently engaged in the game and to begin to circumscribe
the demographics and clinical characteristics of the users for whom the digital therapy
can have its highest impact. The Superpower Glass program started with co-design experiments
which examined how children from different demographics and a wide range of autism
severity levels interacted with our device.[24]
[32] Likewise, the clinical development of AKL-T01 began with a feasibility study of
the effectiveness of a prototype of the intervention in a wide range of children with
sensory processing dysfunction with and without ADHD.[33] Following this exploratory stage, to settle on the core functionalities of the digital
intervention, researchers must pilot test the intervention's utility and potential
in a treatment-only clinical feasibility test. A feasibility test of the wearable
augmented reality Superpower Glass system showed significant changes in meaningful
clinical end points in 14 autistic children of an average age of 7 years 6 months
(SD = 2.51 years).[34] Similarly, AKL-T01 was tested in a study which allowed the researchers to decide
on a final clinical target and a set of product characteristics that showed the greatest
potential for therapeutic benefit among their target patients.[35] A key next step, often a final step to commercial pathways, is to test the effect
of the digital therapeutic in a formal randomized controlled trial that follows an
intention-to-treat trial protocol. Both Superpower Glass[26] and AKL-T01[28] were tested in randomized controlled trials and showed significant positive changes
on their primary endpoint measures in the treatment group.
We have followed our digital therapeutic translational framework outlined above to
test the feasibility of GuessWhat's computer–human interaction design, user experience and engagement in laboratory[36] and home environments[37]
[38]
[39] and to optimize the system to be both fun and therapeutic. Specifically, we prototyped
the game with eight families over 6 weeks with a combination of in-person focus group
sessions, beta testing, and at-home game play. We then launched the application on
the Android and iOS application stores for free download and use and marketed the
game's availability through Facebook and other social media outlets to encourage players
(irrespective of autism diagnosis or any other characteristic) to try the game. We
gathered play metrics and electronic feedback from this virtual collection of players
to further refine the game design. We evaluated the data feed to settle on the 90-second
game session timeframe for the charades challenges as optimal both for engagement
and for data acquisition. These aspects of the human computer interaction design led
to the version tested here, which was optimized for engagement and treatment potential.
In the present study, we test this version of the game's feasibility as an at-home
digital therapeutic for children with autism between the ages of 3 and 12 years (the
current ages where we have performed user-testing and engagement design iterations).
This study is an important and necessary step toward a formal randomized controlled
trial.
Methods
Study Design and Participants
This study was conducted remotely. Participants were recruited through GuessWhat's existing userbase, The Hartwell Foundation's KidsFirst autism research database, ResearchMatch.org, and Facebook advertisements. All participating
families met the following criteria: (1) able to read and speak English, (2) possessed
or had access to a compatible iOS or Android device with internet access, (3) the
parent was 18+ years old, (4) the child was between 3 and 12 years of age and diagnosed
with ASD. To safeguard against the potential for self-reporting bias, we required
the caregiver to confirm that their child's autism diagnosis came from a formal medical
assessment. We asked the caregiver to choose a diagnostic label from a menu of choices
including ASD, autistic disorder, pervasive developmental disorder-not otherwise specified,
Asperger syndrome, ADHD/ADD, anxiety, speech and language delay. In addition, we required
participants to report on the specific type(s) of therapy being administered to their
child. We also asked participants to fill out fields associated with the Mobile Autism
Risk Assessment,[40] a machine learning tool validated to have high accuracy for autism screening. Finally,
our clinical team reviewed all videos shared by participants for exhibition by the
autistic child of symptoms and features consistent with an autism diagnosis. All participants
provided informed consent online on behalf of their child prior to participation.
Ethical approval was obtained from Stanford University's Institutional Review Board
prior to all research activities. This approval included a Data Risk Assessment performed
by the University Privacy Office that found our system to be compliant with Stanford
University, State of California, and Federal privacy regulations.
Mobile Game Platform
GuessWhat is a mobile application available for at-home use through any iOS and Android device
([Fig. 1A]). The game uses a form of mobile charades to create opportunities for social exchange
between a child and a playing partner centered around face and eye contact and expanded
range of emotion recognition and expressivity for the child. A caregiver holds a smartphone
on their forehead while prompts appear on the screen. The child must act out the prompts
so that the caregiver will be able to guess what the child is acting ([Fig. 1B, C]). The gameplay integrates key components of applied behavior analysis (ABA),[41] discrete trial training,[42] and pivotal response training[43] ([Table 1]) to engage children in self-initiated and naturalistic imitation[44] and sociodramatic and symbolic play[45] with their social partner (e.g., their parent). In-app daily, weekly, and monthly
challenges and reward mechanisms (including audio affirmation, coins, and the unlocking
of new game decks and achievement badges) reinforce positive progression through the
game and maintain the child's interest through time ([Fig. 1], day f).
Fig. 1 The GuessWhat Mobile User Experience. (A) GuessWhat is a mobile game available for any smartphone device. In a typical game session,
(B) the parent holds the smartphone to their forehead and tries to guess what the child
is acting in response to the prompt shown on the phone's screen. Upon guessing, the
parent tilts the phone to proceed to the next prompt through the end of the 90-second
session. (C) After each 90-second game, parent and child can review together, enhancing and strengthening
the learning. In-app (D) game modes, (E) unlocking deck and character choices based on coins earned, and (F) activity-based achievement badges reinforce positive progression and ensure optimal
child engagement through time.
Table 1
GuessWhat game elements and treatment focus areas. The parent holds the phone on forehead,
variable image prompts appear on screen, and child acts what is displayed ([Fig. 1]). If parent can guess the prompt correctly, they tilt the phone downward to earn
a coin and move to the next prompt. If unable to guess, the parent tilts the phone
upward to skip to the next prompt. The game in its current form comes with over 40
different deck choices that can be used alone or mixed into sessions lasting 90 seconds
each. Earned coins enable unlocking of new decks and the purchase of new game characters
desired by the child. The mobile application is available on Android and iOS; both
versions are linked from guesswhat.stanford.edu
|
GuessWhat game element
|
Treatment focus area
|
|
Charades: This is a primary game mode. Act out images displayed on screen. Vary the
acting challenges and try to score higher with each session.
|
Eye contact, joint attention, theory of mind, imitation, expressive and receptive
communication, nonverbal social awareness, conceptual (abstract) reasoning; flexible
thinking.
|
|
Selecting and mixing decks: Choose from 40 different decks that feature different
animals, sports, occupations, environments, and more.
|
Self-initiation, independence, comprehension of and response to multiple cues, reinforcement
of variable interests/behaviors.
|
|
Earning coins: Correct guesses by play partner rewarded with coins that allow purchase
and unlocking of new game decks and game characters.
|
Social motivation
|
|
Turn taking mode: Child and parent prompted to control the phone and take turns acting
out the challenges displayed on the phone screen.
|
Turn taking, sharing, social initiations
|
|
Video playback: Watch yourself perform the challenge.
|
Reinforcement learning and course correction
|
|
Motion challenges: Sports, gestures, objects, chores decks.
|
Gross and fine motor skills
|
|
Sound challenges: sing-along, pledge allegiance, poetry recitals.
|
Speech and language
|
|
Emotion challenges: emojis, human faces decks, cartoon character faces.
|
Facial emotion recognition and expression
|
Each play session begins by the parent selecting one or more of several prompt decks,
such as Faces, Dances, or Emojis, which aim at core behavioral targets such as receptive and expressive communication,
joint attention, multiple cue response, gross and fine motor skills ([Table 1]).[9]
[41]
[42]
[43] Once a game deck(s) has been selected, the parent presses “Play” and holds the phone
to their forehead with the screen facing the child. The child then interprets the
prompt displayed on the screen and acts out what they see (e.g., “surprise” for the
Faces deck or “the floss” for the Dances deck). The parent must then guess what the child is acting out and confirm if they
are correct by communicating via verbal or social cues and tilting the phone up or
down. GuessWhat uses the phone's accelerometer to detect a tilt in the forward direction indicating
a correct guess or a tilt back indicating “skip” to proceed forward to the next prompt.
After 90 seconds, the game session is complete, and the results are presented back
to the user as how many prompts were guessed correctly versus skipped. The parent
is then given the opportunity to review the answers and the gameplay video with the
child, and to save the video to their phone, share it with our laboratory, or delete
it altogether. When a user chooses to share their gameplay video, it is stored in
a Health Insurance Portability and Accountability Act (HIPAA) compliant encrypted
cloud database for further research.
Participant Procedures
All participants were asked to download and start playing GuessWhat at the beginning of the study. Families were instructed to play the game for at least
three 90-second sessions per day, 3 days per week, for 4 weeks, for a total of 54 minutes
of gameplay over 1 month. Participants were directed to play any of the available
game decks, with particular emphasis on Faces and Emojis, which were specifically designed to encourage emotion recognition and social communication.
Parent-reported surveys were collected for all participants immediately before and
after the 4-week testing period. Participants were sent an Amazon gift card for the
completion of study procedures.
Measures
Gameplay metrics and qualitative feedback were collected from all participants who
used the GuessWhat application. Gameplay metrics were collected throughout the 4-week intervention period
and included game deck selection, number of 90-second game sessions initiated and
completed, and total play time per session, per login, and for the entire study period.
A qualitative feedback measure created by our research team assessed the GuessWhat players' user experience at the end of the 4-week period. This measure includes 11
free-text and multiple-choice items capturing participants' feedback and suggestions,
technical difficulties they experienced while playing the app, preferred game modes,
and likelihood of using the app beyond the study.
All study participants responded to two standardized clinical outcome measures for
autism, the Social Responsiveness Scale, 2nd edition[46] (SRS-2) and the Vineland Adaptive Behavior Scales, 2nd edition[47] (VABS-II). Both surveys were collected at the start and end of each family's 1-month
testing window.
SRS-2 is a 65-item survey intended to identify the presence and severity of social
impairment in children across five social domains. Standardized scores at or above
60 are considered indicative of social impairment.
VABS-II is a 502-item survey which measures children's adaptive functioning on four
main domains: communication, daily living skills, socialization, and motor skills.
We collected the Socialization Domain (99 questions that measure interpersonal relationships,
play and leisure time, and coping skills) and the Receptive and Expressive Communication
subscales (74 items) of the Communication Domain using the Comprehensive Parent/Caregiver
Form of the survey. Higher scores indicate greater social functioning.
Analysis
Usage metrics were analyzed across each participant from first date of gameplay following
pre-test survey data collection through last gameplay session prior to post-test survey
data collection. Free-text responses in the qualitative feedback measure recorded
during post-test survey data collection were evaluated using content analysis to identify
the most common technical difficulties as well as overall satisfaction and engagement
with the GuessWhat mobile game. Analysis of the qualitative feedback measure also included the quantification
of multiple-choice radio select options measuring how many participants intended to
continue playing the app following study completion.
Changes in SRS-2 and VABS-II scores were assessed using means comparisons with two-tailed
paired sample t-tests. Values 1.5 times above or below the interquartile range were excluded from
the analysis. Families who did not record GuessWhat play attempts but completed all other procedures were analyzed as a separate group
for comparison.
Results
We tested the therapeutic feasibility of our mobile game with 72 autistic children
(75% male, average age 8 years 2 months, minimum age 3 years 10 months, maximum age
12 years 6 months) ([Fig. 2]). Sixty-two (86%) players were receiving standard autism therapy treatments, with
60 (83%) receiving two types or more during the study ([Table 2]). A comparison control group that did not play the game but completed all other
procedures included 19 children (84% male, average age 8 years 5 months, minimum age
3 years 11 months, maximum 12 years 4 months). Eighteen (95%) children in the comparison
group received some form of autism-related therapy, with 13 (68%) receiving two types
or more ([Table 2], [Supplementary Fig. S1], available in the online version). Prior to the start of our GuessWhat treatment,
we did not find significant differences between the two groups in the two baseline
measures, the Social Responsiveness Scale (SRS-2) and the Vineland Adaptive Behavior
Socialization Standard Score ([Table 2]). We did not find any disagreements between our independent confirmation of the
autism diagnosis and the self-reported autism diagnosis in either the treated or the
comparison group.
Fig. 2 Study groups. Ninety-one families were assessed for eligibility and completed pre-
and post-test surveys between March 2019 and December 2020. Seventy-two families played
GuessWhat as instructed. Nineteen families who did not record GuessWhat play attempts but fulfilled all other requirements were analyzed as a separate group
for comparison.
Table 2
User demographics and survey scores for the 72 autism families in the GuessWhat treatment group and the 19 matched participants in the comparison control group (autism
families who did not play GuessWhat but completed all other procedures)
|
Demographic information, Mean (SD)/Percent (N)iii
|
All users (N = 72)
|
Users with ≥28 sessions (N = 17)
|
Users with 4 wk of consistent usage (N = 13)
|
Comparison group (N = 19)
|
|
Age (years)
|
8.17 (2.25)
|
7.97 (2.58)
|
7.9 (2.13)
|
8.4 (2.89)
|
|
Gender (% male)
|
75% (54)
|
80% (16)
|
85% (11)
|
84% (16)
|
|
Comorbidity
|
Attention deficit disorder
|
14% (10)
|
15% (3)
|
8% (1)
|
21% (4)
|
|
Anxiety disorder
|
6% (4)
|
0% (0)
|
0% (0)
|
21% (4)
|
|
Bipolar disorder
|
1% (1)
|
0% (0)
|
8% (1)
|
0% (0)
|
|
Sensory processing disorder
|
4% (3)
|
0% (0)
|
0% (0)
|
5% (1)
|
|
Race and
ethnicity
|
Caucasian/Euro-American
|
72% (52)
|
65% (13)
|
77% (10)
|
68% (13)
|
|
African American
|
7% (5)
|
10% (2)
|
0% (0)
|
26% (5)
|
|
East Asian/Asian American
|
10% (7)
|
10% (2)
|
8% (1)
|
0% (0)
|
|
South Asian/Indian American
|
4% (3)
|
10% (2)
|
0% (0)
|
5% (1)
|
|
Middle Eastern/Arab American
|
1% (1)
|
0% (0)
|
0% (0)
|
0% (0)
|
|
Pacific Islander
|
0% (0)
|
0% (0)
|
0% (0)
|
11% (2)
|
|
Native American/Alaskan Native
|
1% (1)
|
0% (0)
|
0% (0)
|
5% (1)
|
|
Hispanic/Latino/Spanish Origin
|
22% (16)
|
20% (4)
|
8% (1)
|
21% (4)
|
|
Unknown/Not listed
|
6% (4)
|
5% (1)
|
77% (10)
|
0% (0)
|
|
Concurrent therapy enrollment
|
Applied behavior analysis (ABA)
|
47% (34)
|
50% (10)
|
62% (8)
|
58% (11)
|
|
Social skills therapy
|
35% (25)
|
40% (8)
|
31% (4)
|
37% (7)
|
|
Special education classes
|
44% (32)
|
30% (6)
|
38% (5)
|
53% (10)
|
|
Speech-language pathology
|
74% (53)
|
70% (14)
|
69% (9)
|
47% (9)
|
|
Occupational therapy
|
58% (42)
|
65% (13)
|
62% (8)
|
68% (13)
|
|
No therapy
|
14% (10)
|
15% (3)
|
8% (1)
|
5% (1)
|
|
1 type of therapy only
|
2.8% (2)
|
5% (1)
|
0% (0)
|
26% (5)
|
|
2+ types of therapy
|
83% (60)
|
80% (16)
|
0% (0)
|
68% (13)
|
|
Clinical evaluations
|
|
|
Social Responsiveness Scale (SRS-2), intake score
|
Social awareness
|
75.24 (9.85)
|
74.29 (9.42)
|
76.54 (6.71)
|
79.71 (10.44)
|
|
Social cognition
|
77.46 (8.36)
|
76.47 (9.07)
|
77.77 (6.1)
|
76.22 (11.63)
|
|
Social communication
|
78.5 (9.51)
|
78.53 (11.23)
|
80.08 (4.05)
|
79.24 (9.29)
|
|
Social motivation
|
69.67 (11.14)
|
68.12 (11.94)
|
67.62 (8.71)
|
70.4 (5.74)
|
|
Restricted and repetitive behavior
|
79.04 (9.77)
|
78.24 (9.97)
|
82.31 (7.03)
|
77 (11.06)
|
|
Total*
|
80 (8.43)
|
78.82 (10.56)
|
81 (4.56)
|
80.82 (8.18)
|
|
Vineland adaptive behavior scales-II (VABS-II), intake score
|
Receptive communication
|
9.55 (3.31)
|
9.76 (3.88)
|
8.85 (1.63)
|
8.93 (2.43)
|
|
Expressive communication
|
9.86 (4.04)
|
10.94 (5.02)
|
9 (1.28)
|
8.22 (4.05)
|
|
Socialization relationships
|
8.99 (3.69)
|
9.56 (3.79)
|
8.54 (2.02)
|
7.78 (3.25)
|
|
Socialization play
|
8.55 (3.18)
|
10.18 (5.87)
|
8.77 (2.3)
|
7.67 (4.56)
|
|
Socialization adaptive
|
10.82 (3.21)
|
12 (3.82)
|
10.23 (2.13)
|
9.65 (3.14)
|
|
Socialization standard Total*
|
68.94 (15.53)
|
73.44 (20.85)
|
68.08 (6.75)
|
65.17 (20.52)
|
Note: The baseline total scores (designated with a) for the two primary measures did not differ significantly between the treatment
and control groups, regardless of usage patterns.
All players reported high engagement and satisfaction with the GuessWhat mobile application. Sixty-seven percent of families expressed intention to continue
playing beyond the study period. Eleven families reported experiencing technical difficulties,
five of whom were able to resolve the issues during the testing period. The most common
issues experienced by these users were trouble logging in (n = 5), poor internet connection (n = 3), or the application freezing or crashing (n = 2). Thirteen families reported difficulties such as confusion about how to play
the game (n = 6) or prompts that were too difficult (n = 4). Seven families expressed difficulty interpreting the game instructions.
GuessWhat users played an average of 17.68 (SD = 18.04) sessions over 5.14 (SD = 4.63) days
during the 4-week testing period, 49% of the recommended use. Participants selected
the Emojis deck in 16% of the game sessions, Faces in 12%, Animals in 13%, followed by Objects (8%), Jobs (7%), Sports (7%), and all others (37%) ([Fig. 3]). Proper gameplay was verified through videos recorded during game play and that
were shared with our team ([Table 3]).
Fig. 3
GuessWhat game decks. Users played a total of 1,965 game decks over the course of the study.
The Emojis deck was chosen in 16% of all initiated game sessions, followed by Animals (13%), Faces (12%), Objects (8%), Jobs (7%), and Sports (7%). Cumulatively, all other decks were chosen in 37% of game sessions.
Table 3
GuessWhat usage statistics. Gameplay statistics were recorded over 1 mo for all 72
players in the treatment group
|
Game start
|
Game end
|
Game share
|
Points
|
|
Game
|
Num.
|
Avg.
|
Num.
|
Avg.
|
Num.
|
Avg.
|
Num.
|
Avg.
|
|
Emojis
|
316 (16%)
|
4.33 (SD = 5.98)
|
251 (15%)
|
3.44 (SD = 4.68)
|
208 (14%)
|
2.85 (SD = 4.1)
|
2,220 (25%)
|
30.41 (SD = 50.44)
|
|
Faces
|
233 (12%)
|
3.19 (SD = 4.34)
|
201 (12%)
|
2.75 (SD = 4.11)
|
172 (12%)
|
2.36 (SD = 3.82)
|
1,950 (22%)
|
26.71 (SD = 50.51)
|
|
Animals
|
250 (13%)
|
3.42 (SD = 7.15)
|
223 (13%)
|
3.05 (SD = 6.57)
|
202 (14%)
|
2.77 (SD = 6.55)
|
1,818 (21%)
|
24.90 (SD = 52.49)
|
|
Objects
|
155 (8%)
|
2.12 (SD = 4.09)
|
136 (8%)
|
1.86 (SD = 3.52)
|
111 (8%)
|
1.52 (SD = 3.34)
|
884 (10%)
|
12.11 (SD = 27.52)
|
|
Jobs
|
136 (7%)
|
1.86 (SD = 5.05)
|
124 (7%)
|
1.70 (SD = 4.54)
|
103 (7%)
|
1.41 (SD = 4.32)
|
720 (8%)
|
9.86 (SD = 27.37)
|
|
Sports
|
134 (7%)
|
1.84 (SD = 5.07)
|
113 (7%)
|
1.55 (SD = 4.58)
|
93 (6%)
|
1.27 (SD = 4.43)
|
746 (8%)
|
10.22 (SD = 25.88)
|
|
Other
|
740 (38%)
|
10.15 (SD = 24.81)
|
665(39%)
|
9.11 (SD = 21.95)
|
591 (40%)
|
8.10 (SD = 21.55)
|
488 (6%)
|
6.68 (SD = 95.66)
|
|
Тotal decks
|
1,964
|
26.92 (SD = 47.08)
|
1713
|
23.47 (SD = 42.46)
|
1,480
|
20.27 (SD = 41.32)
|
8,826
|
120.90 (SD = 158.86)
|
|
Total sessions
|
1,291
|
17.68 (SD = 18.04)
|
1105
|
15.14 (SD = 15.81)
|
959
|
13.14 (SD = 15.32)
|
183.93 (N/A)
|
Users demonstrated an average of 3.97-point improvement in SRS-2 total T-score (SD = 4.92,
p < 0.001) as well as an improvement of 5.27 points on the VABS-II socialization standard
score (SD = 9.29, p = 0.002). The game players who played more than 28 sessions (n = 17) showed a 3.82-point (SD = 5.70, p = 0.01) positive change on the SRS-2 Total T-score and a 6.21-point (SD = 8.13, p = 0.11) improvement on the VABS-II Socialization Standard score. The game players
who played consistently over all 4 weeks (n = 13) showed a 2.85-point (SD = 4.56, p = 0.03) positive change on the SRS-2 total T-score and 5.46-point (SD = 6.75, p = 0.58) improvement on the VABS-II socialization standard score. The 19 families
who did not play GuessWhat but completed all other procedures showed no significant change in scores recorded
by either survey. Results of the primary means comparisons are presented in [Table 4] (subsection scores available in [Supplementary Table S1], available in the online version).
Table 4
Change in parent-reported survey scores by user group. Results include 72 families
who played GuessWhat and a comparison group of 19 matched participants who did not play GuessWhat but completed all other procedures. Reductions in SRS-2 and increases in VABS-II
scores are indicative of improvement and vice versa
|
Measure
|
Mean change from pre-test to post-test (SD)
|
|
All treated players
|
p-Value
|
Users treated with ≥28 sessions
|
p-Value
|
Users with 4 wk. treatment consistency
|
p-Value
|
Comparison group
|
p-Value
|
|
SRS-2 total T-score
|
−3.97 (SD = 4.92)
|
<0.001[c]
|
−3.82 (SD = 5.70)
|
0.01[a]
|
−2.85 (SD = 4.56)
|
0.03[a]
|
−4.71 (SD = 7.92)
|
0.10
|
|
VABS-II socialization standard score
|
5.27 (SD = 9.29)
|
0.002[b]
|
6.21 (SD = 8.13)
|
0.11
|
5.46 (SD = 6.75)
|
0.58
|
−1.61 (SD = 12.37)
|
0.62
|
a p <0.05.
b
p <0.01.
c
p <0.001 between cohorts from a two-tailed paired sample t-test.
Discussion
Our study examined the potential for the mobile game GuessWhat to provide a therapeutic effect for autistic children and families who play the game
for a period of 1 month. The game fosters a prosocial exchange between the child and
playing partner in short 90-second sessions designed for social skills development
including eye contact and emotion recognition. Out of the 91 families assessed for
eligibility who completed pre- and post-test surveys, 72 (79%) successfully played
one or more GuessWhat game sessions. The 19 (21%) families who did not play the game at all were assigned
to a separate comparison group. The 72 GuessWhat users exhibited significant improvements in SRS-2 and VABS-II scores, while the untreated
comparison group showed no significant differences in either measure over the same
4-week timeframe.
Average improvements in SRS-2 total scores were in line with minimal clinically important
differences (MCIDs) required for a shift from severe to moderate, moderate to mild,
or mild autism severity to within normal limits.[48] Average improvements in VABS-II Socialization domain scores exceeded the MCID of
3.7 points.[49] Interestingly, these therapeutic effects were evident even with adherence to dose
recommendations at approximately 50% of the recommended game play frequency. GuessWhat users averaged 5.14 gameplay days over the 4-week testing period even though they
were asked to play a minimum of 12 days (three sessions, 3 days a week over 4 weeks).
Potential reasons for the lower adherence include technical difficulties experienced
by some families, decreased child engagement with the app over time due to boredom,
and/or decreased motivation by the parent to continue using the intervention. While
these lower usage patterns demonstrate the need for an enhanced user interface including
reminder and reward systems, the observed gains in socialization provide support for
the digital game's potential as a useful form of autism therapy. The results also
support previous game therapy research showing that parental involvement, a major
design focus for our system, positively impacts the therapeutic effect.[50]
Players who played more than 28 GuessWhat sessions and/or played during each week of the intervention period showed larger
improvements on VABS-II Socialization Standard scores and on VABS-II Receptive and
Expressive Communication scores suggesting that improvements may be greater with higher
compliance to the recommended gameplay ([Table 4]). This dose-dependency in therapeutic gains also supports the likelihood that the
observed effects were likely not due to a placebo effect. Further tests are needed
to confirm the therapeutic effectiveness, to determine optimal levels of gameplay,
to design the most effective means of ensuring adherence to it, and to show the potential
for sustained gains after therapy periods conclude.
Limitations
This research demonstrates how a new type of mobile game-based therapy may be used
from the convenience of people's homes for improving social acuity in children with
autism. One limitation of the study was the use of nonblinded, parent-administered
intervention and outcome measures, which risks biased reporting. It can be expected
that potential biases in parent reporting would manifest as stronger on the SRS-2
scale than on VABS-II, since the latter asks more quantitative questions and therefore
leaves less opportunity for subjective judgment. Accordingly, the lack of discrepancies
between SRS-2 and VABS-II results for GuessWhat players ([Table 4], [Supplementary Table S1], available in the online version) demonstrates that the socialization gains observed
following a month of gameplay were not primarily attributable to biases in parental
reporting. Instead, the game's therapeutic effect is likely due to the increased frequency
of (1) structured social engagement between parent and child, and (2) practice of
facial and emotion recognition skills during the charades-style interaction.
A second limitation of the present study was potential self-selection bias. It may
be the case that the 19 families who did not attempt to play GuessWhat but completed all other study procedures had lower motivation to participate in the
study. A lack of motivation in this comparison group could have therefore resulted
in reduced gains measured by either SRS-2 or VABS-II parent-reported surveys. Nevertheless,
the inconsistent changes between the two outcome measures (SRS-2 scores improved while
VABS-II worsened) and the large standard deviation observed in the comparison group
([Table 4], [Supplementary Table S1] [available in the online version]) indicate that reduced gains due to lack of motivation
did not sufficiently account for the differences observed between GuessWhat players and non-players. A larger comparison group will be needed to confirm these
results.
A third limitation was that the demographics of our study sample were predominantly
male and Caucasian ([Table 2]). Recruitment of balanced populations is challenged by the 4:1 male:female prevalence
of autism[1] and underdiagnosis and undertreatment among non-Caucasian autism families.[2] Simple correlations between participant demographics and usage data or outcome measures
yielded no clear conclusions about the best responders. We will use GuessWhat's advantages as a digital intervention to develop new game decks that will appeal to
participants of any gender or ethnicity and to recruit a more balanced cohort among
a wider range of participants from all over the United States and the world in future
studies.
Finally, the enrollment of 32 GuessWhat treatment and 18 of the 19 control participants during COVID-19-related school closures
may have introduced additional confounding factors such as loss of regular therapy,
childcare and school services, and others. Changes in therapy and lifestyle during
the intervention period or at a post-test remain a topic for future investigation.
Conclusion
This study demonstrates that GuessWhat can improve socialization in children with autism in an engaging and accessible manner.
The results support the potential of game-based mobile systems to augment the standard
approaches to autism therapy, ideally increasing the continuity of behavioral therapy
and preventing loss in services where access to care is limited.
Clinical Relevance Statement
Clinical Relevance Statement
The results of this study indicate that game-based digital therapeutics may be an
effective to deliver behavioral therapy for children with autism spectrum disorder.
This outcome is particularly relevant in the context of the current coronavirus disease
pandemic and restrictions to in-person clinical services. Mobile tools such as GuessWhat present a viable way for bridging gaps in continuity of care and ensuring that all
families can access effective and engaging behavioral therapy from the comfort of
their home.
Multiple Choice Questions
Multiple Choice Questions
-
What is the first step (after prototyping) in the design of a novel digital therapeutic?
Correct Answer: The correct answer is option c. The design of novel digital therapeutic begins with
a series of co-design experiments done with the target patient population intended
to optimize engagement, to begin to understand the dose–response relationship, to
refine hypotheses regarding the expected clinical treatment effect, and to generally
understand the ways in which the stakeholders will interact with a prototype of the
product.
-
When did FDA begin clearing game-based digital therapeutics for market use?
-
1990s.
-
2000s.
-
2010s.
-
2020s.
Correct Answer: The correct answer is option d. The Food and Drug Administration cleared the first
game-based digital intervention for market use in the United States in June 2020.
