Diagnostic accuracy of CompCog: reaction time as a screening measure for mild cognitive impairment

ABSTRACT

Background Reaction time is affected under different neurological conditions but has not been much investigated considering all types of mild cognitive impairment (MCI). Objective This study investigated the diagnostic accuracy of CompCog, a computerized cognitive screening battery focusing on reaction time measurements. Methods A sample of 52 older adults underwent neuropsychological assessments, including CompCog, and medical appointments, to be classified as a control group or be diagnosed with MCI. The accuracy of CompCog for distinguishing between the two groups was calculated. Results The results from diagnostic accuracy analyses showed that the AUCs of ROC curves were as high as 0.915 (CI 0.837-0.993). The subtest with the highest sensitivity and specificity (choice reaction time subtest) had 91.7% sensitivity and 89.3% specificity. The logistic regression final model correctly classified 92.3% of individuals, with 92.9% specificity and 91.7% sensitivity, and included only four variables from different subtests. Conclusions In summary, the study showed that reaction time assessed through CompCog is a good screening measure to differentiate between normal aging and MCI. Reaction time measurements in milliseconds were more accurate than correct answers. This test can form part of routine clinical tests to achieve the objectives of screening for MCI, indicating further procedures for investigation and diagnosis and planning interventions.

 

MissingLinkText

RESUMO

Antecedentes O tempo de reação é afetado em diferentes condições neurológicas, mas não foi muito investigado considerando todos os tipos de comprometimento cognitivo leve (CCL). Objetivo Este estudo investigou a acurácia diagnóstica do CompCog, uma bateria computadorizada de rastreio cognitivo focada em medidas de tempo de reação. Métodos Uma amostra de 52 idosos passou por uma avaliação neuropsicológica, incluindo o CompCog, e uma consulta médica para serem classificados como grupo controle ou serem diagnósticos com CCL. A acurácia do teste para distinguir entre os dois grupos foi calculada. Resultados Os resultados das análises de acurácia diagnóstica mostraram AUC das curvas ROC tão altas quanto 0,915 (CI 0,837-0,993). O subteste com maior sensibilidade e especificidade - subteste de tempo de reação de escolha - apresentou sensibilidade de 91,7% e especificidade de 89,3%. O modelo final de regressão logística classificou corretamente 92,3% dos indivíduos, com especificidade de 92,9% e sensibilidade de 91,7%, e incluiu apenas 4 variáveis de diferentes subtestes. Conclusões Em resumo, o estudo mostrou que o tempo de reação avaliado pelo CompCog é uma boa medida de rastreio para diferenciar entre envelhecimento normal e CCL. Medidas de tempo de reação em milissegundos se mostraram melhores que o número de respostas corretas. O teste pode fazer parte de testes clínicos de rotina para atingir o objetivo de rastrear o CCL, indicar outros procedimentos para investigação e diagnóstico e planejar intervenções.

 

MissingLinkText

Keywords:

Cognitive Dysfunction - Reaction Time - Diagnosis - Dementia - Cognitive Aging

Palavras-chave:

Disfunção Cognitiva - Tempo de Reação - Diagnóstico - Demência - Envelhecimento Cognitivo

INTRODUCTION

The American Academy of Neurology has acknowledged the utility of diagnosing mild cognitive impairment (MCI) as proposed by Petersen[1]. Its utility is related to the higher rate of conversion of individuals diagnosed with MCI to dementia than among those not diagnosed[2], and to the possibility of implementing early interventions to improve quality of life[3],[4].

The diagnosis of MCI has evolved over the years. Today, it includes subtypes with different etiologies and prognostics[5]. Thus, it is a heterogeneous construct that can involve subtle cognitive impairment of several functions that are not consistently detectable through commonly used screening tests[6]. It is a challenge to detect MCI in its early years, before it has progressed to severer forms of cognitive decline like dementia. Although research is making progress, it is usually more focused on (1) forms of MCI related to Alzheimer’s disease[7],[8] and (2) techniques using technologies that are not always accessible or used in screening processes, like neuroimaging and biomarkers[9],[10].

Evidence suggests that assessment of some cognitive variables may constitute a noninvasive and affordable first step regarding screening for cognitive decline[11],[12]. One cognitive variable that might be affected more homogenously through MCI heterogeneity is processing speed. This can be assessed through several different variables[13]. Response speed and reaction time are the ones most used and can be understood as the time taken by an individual to issue a response after a stimulus[14]. This can be measured as a mean or median from several trials or also by considering intraindividual variability[14]. Although the findings have not been consistent, variability and reaction time are not necessarily impaired or affected simultaneously or with the same severity[14]. Nevertheless, concerning MCI, both measurements are of interest, given that studies have shown that both of them may be impaired[15],[16].

For a long time, processing speed has been seen as a fundamental aspect of cognition and an essential aspect of healthy aging[17]. Studies have shown that it declines through many neurodegenerative conditions[13],[15],[16],[18] ^- [20]. One metanalysis found slower reaction time among MCI patients than among healthily aging individuals[21]. Although the majority of such studies only considered amnestic patients, the same results were found in two studies that also considered non-amnestic MCI patients[22],[23].

Studies have shown that reaction time can decrease before errors start to be committed. Sometimes, a task can be completed, but in more time than usual[24],[25]. However, this variable is not commonly measured, considering that (1) precision is required for detecting changes at the beginning of pathological aging processes[19],[23] and (2) paper-and-pencil neuropsychological assessment predominates[26]. Paper-and-pencil neuropsychological tests rarely involve precise reaction time measurements that can detect the subtle changes in the first stages of pathological aging[27],[28]. However, one option for addressing this matter is to use computerized tests. These can be successful in this task because they provide precise reaction time measurements[29].

Use of computerized tests also brings other benefits, such as greater control over the administration and scoring of tests, reduction of errors in scoring and reduction of examiner’s bias[26]. This is especially true for low and middle-income countries, where resources are limited and there is a need for fast and cheap methods that are amenable to large-scale administration[10].

In the present study, we investigated whether the reaction time measurements of the CompCog computerized battery are helpful for discriminating between MCI and healthy individuals. This battery uses an iPad interface, and all responses are issued using a touchscreen. During each test, the type of response and reaction time in milliseconds are recorded. A previous version of the same test is already known to distinguish between healthy individuals and individuals with Alzheimer's disease[30]. Thus, CompCog was expected to form a valuable tool for detecting MCI.

 

MissingLinkText

METHODS

Setting and procedures

Participants were invited after involvement in a larger study conducted in partnership with a social program offered by Rio de Janeiro's government[31]. This program provides daily activities for older adults during the day, such as physical exercises, stretching, yoga, dance, cognitive stimulation, crafts, theater, etc. The first study evaluated older adults through a brief neuropsychological assessment done by researchers and senior neuropsychologists. All psychologists attended weekly supervision with the coordinator of the Applied Psychology Service of the Pontifical Catholic University of Rio de Janeiro. The evaluation lasted one hour and was held in a quiet room in the houses where the social program commonly took place. During the assessment, cognitive tests and scales were used to assess cognition, depressive symptoms and functionality. These are all described in the corresponding section below.

Participants in the larger study were randomly invited to join the present study. The ones who accepted this underwent another neuropsychological testing session and a medical appointment with a doctor, at which diagnoses were given. The neuropsychological assessment consisted of (1) a new anamnesis to confirm the clinical and sociodemographic characteristics of the individuals, and the inclusion and exclusion criteria for recruitment; and (2) administration of CompCog. The average session duration was 1h15, and the sessions were carried out at the Applied Psychology Service of the Pontifical Catholic University of Rio de Janeiro. Medical appointments aimed at making diagnoses were conducted at the same place or in the outpatient clinics of the Department of Medicine of the same university.

Geriatricians evaluated the cases and made the diagnoses during medical appointments. The diagnoses were based on clinical history, neuroimaging when available and the initial neuropsychological protocol. This protocol included the following tests and scales: 1) Mini-Mental State Examination (MMSE)[32]; 2) Brief Cognitive Screening Battery[31],[33] consisting of the following tests: Figure Memory Test (MFT), Categorical Verbal Fluency Test (VF) and Clock Drawing Test (CDT); 3) Geriatric Depression Scale (GDS-15)[34]; 4) Functional Activities Questionnaire (FAQ)[35] and 5) Lawton Instrumental Activities of Daily Living Scale[36]. Although the FAQ formed part of the evaluation, it was not used in the analysis because of a high rate of missing data. The maximum interval between the first evaluation and the medical appointment was six months.

 

MissingLinkText

Participants

Seventy older adults (above 60 years old) were recruited for this study. Among them, 40 were classified as healthy older adults, i.e. individuals with no changes in cognitive performance tests and without functional impairment. The other 30 were diagnosed as older adults with MCI. Exclusion criteria eliminated six individuals from the MCI group and two individuals from the control group (CG). The exclusion criteria were the following: (1) presentation of conditions other than MCI that affect cognition (e.g. stroke); (2) recent history of alcohol or other drug dependence; (3) high levels of depressive symptoms, assessed from the score on the depression scale; (4) presence of visual or hearing disorders without correction; (5) illiteracy; and/or (6) use of medications that could affect reaction time (e.g. benzodiazepines). In the CG, 10 cases were randomly excluded until the variables of number of years of education, sex, age, number of health issues, depressive symptoms and number of medications in use had become matched with those of individuals in the MCI group. The resulting sample consisted of 24 participants with MCI and 28 individuals in the CG. The mean age of the MCI group was 73.9 years (6.9); the mean number of years of education was 11.6 (5.3); and 70.8% were women. The mean age of the CG was 71.4 years (5.7); the mean number of years of education was 14.1 (3.3); and 82.1% were women.

Although the diagnosis did not include the MCI type, it was possible to propose a classification into amnestic or non-amnestic based on the paper-and-pencil tests used, i.e. the tests in the Brief Cognitive Screening Battery. Out of the 24 MCI participants, 13 had at least one Z score below -1 in the memory test, and therefore these individuals could be classified as presenting an amnestic MCI type. The other 11 participants did not have Z scores below -1 in the memory test, and therefore could be classified as presenting a non-amnestic MCI type. We consider that these data were insufficient to classify the amnestic or non-amnestic types as multi-domain or single-domain, because more extensive assessments might have shown more deficits[37]. Nevertheless, the differences between the control group and the MCI group are described in the results.

 

MissingLinkText

Instrument

CompCog is a computerized cognitive screening battery with eight subtests that evaluate different cognitive domains: Simple Reaction Time (SRT), Choice Reaction Time (CRT), Implicit Learning Test (ILT), Visual and Spatial Short-Term Memory (STM), Face Recognition and Memory (FRM), Inhibitory Control Test (ICT), Stroop Test (StT) and Survey Test (ST). The subtests are usually presented in this order but can also be randomized. In our study, we used the standard test order. Each subtest is explained in [Table 1] with the respective variables evaluated (52 in total). All responses are issued using a touch screen and recorded. All tests generate reaction time measurements registered in milliseconds for each touch and are presented as the total time and median time, in order to eliminate possible discrepant data from each test.

Table 1 CompCog tests and variables.

Test	Cognitive functions involved and how they are evaluated	Variables
Simple Reaction Time (SRT)	Processing speed. As soon as a white square appears in the middle of the screen, the person should touch the rectangle at the bottom of the screen.	Median reaction time
Choice Reaction Time (CRT)	Processing speed. As a white or orange square appears in the middle of the screen, the person should touch the rectangle of the same color at the bottom of the screen.	Median reaction time; Correct responses; Revised median reaction time (choice reaction time - simple reaction time).
Implicit Learning Test (ILT)	Implicit learning. As one of ten gray squares distributed in the screen turns white, the person should press it. There is a fixed sequence of 25 squares that is repeated four times and one last random sequence.	Median reaction time in each of five tasks; Implicit learning (median reaction time in sequence 4/median reaction time in sequence 1).
Visual and Spatial Short-Term Memory (STM)	Working memory. There are ten gray squares distributed on the screen. One will become white at a time, making a sequence that should be reproduced.	Correct responses; Direct order SPAN; Median reaction time in direct order; Inverse order SPAN; Median reaction time in inverse order.
Face Recognition and Memory (FRM)	Episodic memory. Ten drawings of unknown faces are presented for 30 seconds. The participant should then choose from among ten pairs of faces, the one that was among those initially shown for memorization, in four attempts.	Correct responses and median reaction time for each of the four tasks and for all tasks together.
Inhibitory Control Test (ICT)	Attention and inhibitory control. Squares of different colors will appear in the middle of the screen for one second each: the white ones should be avoided.	Median reaction time; Correct responses; Median reaction time for correct responses; Median reaction time for errors; Errors.
Stroop Test (StT)	Attention and inhibitory control. All tasks have four colored rectangles located at the bottom of the screen. The person should touch the one matching the stimulus that appears in the middle of the screen considering its color without distracters (task 1) and with distracters (tasks 2 and 3).	Interference; Median reaction time and errors for each of the three tasks.
Survey Test (ST)	Attention. Squares of different colors will appear in the middle of the screen for one second each. Participants should press the white ones in the first task, whites and blues in the second and also yellow ones in the third.	Median reaction time, correct responses, reaction time for correct responses, errors and reaction time for errors, for each of the three tasks.

Furthermore, correct response percentages, errors and differences in reaction time between errors and correct responses are also registered. All the stimulus tests are visuospatial, except for one test: the Stroop Test, which contains written words to maintain the original paradigm[38]. With two exceptions, all reaction time medians are calculated after more than 50 trials, with a maximum of 100 trials. The FRM test has a total of 40 trials total, and STM test trials depend on correct responses, with a maximum of 105 trials.

A previous version of the same test is already known to distinguish between healthy individuals and individuals with Alzheimer's disease[30]. Previous analyses regarding the current version showed (1) good construct validity in a principal component analysis, in which variables clustered in agreement with the subtest divisions; and (2) good concurrent validity, with moderate and strong correlations between the CompCog tasks and their equivalents in paper-and-pencil tests[39].

 

MissingLinkText

Ethics

The National Commission for Research Ethics approved this study (opinion no. 965.264; CAAE: 39381514.3.0000.5285) through the UNIRIO Research Ethics Committee. Individuals participated in the study through signing a free and informed consent statement that had been drawn up in accordance with resolution 196/96 of Brazil's National Health Council, which deals with guidelines and standards for research involving human individuals. Participation in this survey was voluntary and the participants did not receive any payment. The study did not bring any risk to the participants' health and they could refuse and/or withdraw consent to participate in the study at any time.

 

MissingLinkText

Statistical analysis

All analyses were conducted using the Statistical Package for the Social Sciences (SPSS, version 22). After verifying through Shapiro-Wilk tests whether the data were normally distributed, differences between groups were tested using t tests for normal distributions or Mann-Whitney tests for non-normal distributions. A chi-square test was used in the case of sex. Receiver operating characteristic (ROC) analysis was performed for each CompCog variable. ROC curves were plotted in order to determine the degree to which subtests discriminated between controls and MCI. As proposed in a recent meta-analysis[6], sensitivity was prioritized instead of specificity since we were offering a screening measure. Therefore, false positives would be better than false negatives, with regard to continuing the clinical investigation. This prioritization was done by choosing the highest sensitivity that still allowed specificity of at least 70%. This method could not be followed regarding five variables for which specificity of at least 70% would cause sensitivity lower than 70%. In those cases, the cutoff point with sensitivity higher than 70% for which the specificity was closest to 70% was chosen.

The variables with higher sensitivity and specificity in ROC analyses were then used in a logistic regression model with the stepwise forward method, to create a model for predicting MCI with the least number of variables. All variables with specificity and sensitivity above 70% were included (24 variables in total). Age and the number of years of education were also included in order to ascertain whether they influenced the model.

 

MissingLinkText

 

MissingLinkText

RESULTS

Sample characteristics

The participants’ performance in neuropsychological assessments and their demographic and clinical characteristics are described in [Table 2]. There was a tendency towards no significant difference between the groups regarding educational level in years (t(37.486) = 2.008; p = 0.052), and there were no significant differences regarding age (t(50) = -1.414; p = 0.164), number of health problems (t(40.60) = -0.0846; p = 0.403), number of medications in use (t(50) = 0.203; p = 0.840), number of depressive symptoms (t(50) = -1.234; p = 0.224) and sex (x² (1) = 0.931; p = 0.335).

Table 2 Clinical characteristics of the sample.

Variable	CG (n = 28) Mean (SD), min-max	MCI (n = 24) Mean (SD), min-max	p-value
Sex*	23/5	17/7	0.335
Age	71.4 (5.7), 62-83	73.9 (6.9), 61-85	0.164
Number of years of education	14.1 (3.3), 6-17	11.6 (5.3), 3-18	0.052
Health problems	1.5 (1.0), 0-3	1.7 (1.4), 0-4	0.403
Medications in use	2.0 (1.9), 0-6	1.9 (2.0), 0-6	0.840
Depressive symptoms	2.8 (1.9), 0-6	3.6 (2.6), 0-9	0.224
Naming	10.0 (0.0), 10-10	9.8 (4.4), 8-10	0.123
Incidental memory	5.9 (1.2), 4-8	5.2 (1.5), 3-8	0.084
Immediate memory**	8.5 (1.1), 6-10	7.2 (1.4), 5-10	0.001
Learning**	9.3 (0.8), 8-10	8.3 (1.1), 6-10	0.001
Delayed recall**	8.6 (1.5), 4-10	7.2 (1.6), 4-10	0.005
Recognition	9.7 (0.4), 9-10	9.5 (0.9), 7-10	0.930
Clock drawing test	6.2 (2.2), 4-10	5.5 (2.5), 1-10	0.259
Verbal fluency**	20.7 (4.3), 13-30	16.6 (4.6), 8-26	0.002
MMSE30	27.0 (2.1), 22-30	25.7 (2.7), 21-29	0.067
Functionality**	20.9 (0.2), 20-21	19.6 (1.6), 15-21	< 0.001

* # female/male; ** significant differences between groups.

There were differences in cognition with regard to immediate memory (t(50) = 3.562; p = 0.001), learning (t(50) = 3.572; p = 0.001), delayed recall (t(50) = 2.914; p = 0.005) and verbal fluency (t(50) = 0.732; p = 0.002). Functionality also differed between the groups (U = 1440; Z = -4.257; p < 0.001). The tests did not show differences regarding naming (U = 308; Z = -1,542; p = 0.123), incidental memory (t(50) = -1.764; p = 0.084), recognition (U = 332.5; Z = -0.087; p = 0.930), clock drawing test (t(50) = 1.141; p = 0.259), and MMSE (t(50) = 1.873; p = 0.067).

ROC curve

The area under the ROC curve (AUC) for all variables can be seen in [Table 3]. For significant variables, the same table shows sensitivity, specificity and cutoff points.

Table 3 AUC for all variables; cutoff points, sensitivity and specificity for significant variables.

Test	AUC, 95% confidence interval range	Cutoff point (milliseconds)	Sensitivity/specificity
Simple Reaction Time Test
MRT	0.506, 0.339-0.673
Choice Reaction Time Test
MRT†	0.915, 0.837-0.993*	689.813	91.7%/89.3%
Correct responses	0.650, 0.498-0.801
Revised MRT	0.705, 0.559-0.852
Implicit Learning Test
MRT 1	0.839, 0.721-0.957*	688.125	83.3%/75%
MRT 2	0.836, 0.720-0.953*	651.531	75%/75%
MRT 3	0.823, 0.703-0.943*	616.159	79.2%/67.9%
MRT 4	0.829, 0.710-0.948*	618.139	75%/71.4%
MRT 5	0.804, 0.674-0.933*	664.784	75%/75%
Implicit learning	0.521, 0.361-0.681
Visual and Spatial Short-Term Memory Test
Correct responses	0.718, 0.577-0.859
Direct order SPAN	0.725, 0.588-0.861
MRT in direct order	0.774, 0.642-0.906*	643.635	75%/71.4%
Inverse order SPAN	0.868, 0.774-0.962*	3.5	95.8%/64.3%
MRT in inverse order	0.781, 0.648-0.915*	668.615	75%/67.9%
Face Recognition and Memory
MRT†	0.896, 0.799-0.993*	1580.791	83.3%/85.7%
MRT 1	0.823, 0.703-0.943*	1905.385	75%/71.4%
MRT 2	0.881, 0.781-0.981*	1506.316	91.7%/75%
MRT 3	0.872, 0.767-0.977*	1430.946	87.5%/75%
MRT 4	0.813, 0.693-0.932*	1486.333	79.2%/71.4%
Correct responses	0.811, 0.685-0.937*	98.750	79.2%/57.1%
Correct responses 1	0.763, 0.628-0.899
Correct responses 2	0.673, 0.537-0.838
Correct responses 3	0.688, 0.537-0.838
Correct responses 4	0.757, 0.619-0.894
Inhibitory Control Test
MRT	0.884, 0.782-0.976*	664.447	87.5%/78.6%
CAMRT	0.871, 0.774-967*	663.279	83.3%/78.6%
EMRT	0.753, 0.615-0.891
Correct responses	0.794, 0.664-0.924*	96.5	75%/75%
Errors	0.794, 0.664-0.924*	3.5	75%/75%
Stroop Test
MRT 1	0.847, 0.732-0.962*	814.839	87.5%/75%
MRT 2	0.799, 0.672-0.926*	911.100	83.3%/67.9%
MRT 3	0.743, 0.604-0.881
Errors 1	0.587, 0.430-0.744
Errors 2	0.541, 0.378-0.704
Errors 3	0.525, 0.336-0.684
Interference	0.506, 0.343-0.669
Survey Test
MRT 1	0.818, 0.703-0.934*	637.844	79.2%/71.4%
Correct responses 1	0.648, 0.497-0.799
CAMRT 1	0.818, 0.703-0.934*	637.844	79.2%/71.4%
Errors 1	0.648, 0.497-0.799
EMRT 1	0.465, 0.307-0.623
MRT 2	0.829, 0.714-0.944*	663.004	79.2%/75%
Correct responses 2	0.798, 0.676-0.919*	97	79.2%/71.4%
CAMRT 2	0.835, 0.721-0.949*	663.004	79.2%/75%
Errors 2	0.798, 0.676-0.919*	3	79.2%/71.4%
EMRT 2	0.573, 0.415-0.731
MRT 3	0.823, 0.707-0.939*	653.629	83.3%/71.4%
Correct responses 3	0.802, 0.678-0.926*	93	75%/75%
CAMRT 3	0.826, 0.711-0.941*	664.223	83.3%/75%
Errors 3	0.802, 0.678-0.926*	7	75%/75%
EMRT 3	0.507, 0.346-0.669

MRT: median reaction time; AUC: area under the ROC curve; CAMRT: median reaction time for correct response; EMRT: median reaction time for error; †best accuracy; *p < 0.001.

In general, reaction time measurements in cognitive tasks of lower complexity (e.g. choosing between colors) and memory tasks were the variables that best discriminated between the CG and MCI group. Simple reaction time, reaction time relating to the Stroop effect, reaction time regarding errors, number of errors and number of correct responses did not differentiate between the CG and MCI group.

 

MissingLinkText

Regression models

The final model correctly classified 92.3% of the individuals, with 92.9% specificity and 91.7% sensitivity, and included four variables. All of these variables concerned reaction time, but in four different tasks: the first task of the Stroop test (odds ratio = 0.979; 95% CI = 0.963-0.996; p = 0.015); the inhibitory control test (odds ratio = 1.027; 95% CI = 1.007-1.048; p = 0.008); the second task of the memory test (odds ratio = 1.009; 95% CI = 1.001-1.017; p = 0.021); and the second sequence of the implicit learning test (odds ratio = 1.018; 95% CI = 1.001-1.036; p = 0.033). Age and the number of years of education did not influence the model. The final model had a chi-square value of 46.183 (4); p < 0.001. The -2 log likelihood was 25.597, with Cox & Snell R-square of 0.589 and Nagelkerke R of 0.786.

 

MissingLinkText

 

MissingLinkText

DISCUSSION

Differences between paper-and-pencil and computerized tests

The first thing to notice is the neuropsychological profile of the sample. Significant differences were found between the groups in paper-and-pencil tests evaluating episodic memory and semantic verbal fluency. The latter has been reported to be highly dependent on semantic memory[40]. Episodic and semantic memory impairments are characteristics of the amnestic subtype of MCI[41]. On the other hand, the reaction times in CompCog tasks involving memory, attention and executive functions showed good accuracy in distinguishing between participants with MCI and the CG.

These results suggest that there is a potential benefit from using computerized tests. These can track a more significant number of impairments than those typically measured through traditional paper-and-pencil assessments. Moreover, with regard specifically to memory performance in CompCog, the accuracy of the number of correct responses was not as high as that of the reaction time.

There are differences between the CompCog memory task and the paper-and-pencil memory test. The CompCog task uses recognition and not recall, as the paper-and-pencil test does. This difference suggests that the CompCog task is easier.

Two benefits can be extracted from this information. The first is the possibility of evaluation without generating performance anxiety and frustration[42], since the numbers of correct responses are similar between the groups. The second is the ability to distinguish between groups before errors start to be committed.

One hypothesis in this regard is that a slower reaction time is one of the first cues of cognitive impairment. Other studies have already shown that the time required for completing tasks increases[25], even before errors hinder their completion[24]. There is also evidence of a correlation between reductions in processing speed and general cognitive performance.[18] It is interesting to note that a reduction in processing speed is also related to subjective memory complaints[43]. Although this kind of complaint usually does not involve an objective deficit in standard tests, it is possible that patients somehow already perceive their slower reaction time. A meta-analysis has suggested that people with subjective memory complaints have twice as high a risk of developing MCI and dementia as do older adults who have no complaints[44]. However, their condition is difficult to measure through traditional memory tasks because individual performances are similar to those of controls[45].

 

MissingLinkText

ROC curve: reaction time is useful as a screening measure for MCI

In general, the ROC curve results showed that reaction time measurements on different cognitive processes were good at distinguishing between healthy individuals and participants with MCI. In comparing these measurements with the numbers of errors and correct responses in the same subtest, the sensitivity and specificity of the reaction time were usually higher, considering reaction times. Normal aging is known to correlate with slower reaction time[13],[18]. However, the results showed signs that the decline might be even more considerable under certain circumstances of pathological conditions, such as in relation to cognitive processes of low and moderate complexity. This conclusion can be drawn from numerous results, but a comparison between the first two subtests might be the clearest: (1) simple reaction time, which was not good at distinguishing the groups; and (2) choice reaction time, which showed the best accuracy, with AUC as high as 0.9.

The results in the literature regarding the topic are mixed. Some studies investigated reaction time in simple tasks and found that this showed good accuracy for distinguishing between participants with MCI and controls[46] ^- [48]. In one study[49], the effect of increasing complexity stimulus was investigated and a division of reaction time into a movement component and a cognitive component was proposed. Activities that solely involved motor reactions, without decision making, could be used to differentiate between patients with Alzheimer’s disease and cognitively healthy old adults, but not between the latter and MCI patients. Only the cognitive component was sensitive to MCI, which suggests that although lower complexity tasks may be useful in this regard, at least some cognitive processing must be involved. This may explain why the Simple Reaction Time test (motor component only) could not distinguish between the groups, but the Choice Reaction Time could, which is a low-complexity cognitive component.

Nevertheless, the same study[49] and others[19],[50] found that more complex variables were better at distinguishing between groups, i.e. a contrary finding. One hypothesis for these contrasting results is that these studies used only the amnestic subtype of MCI. Using just one subtype creates a more homogeneous sample concerning cognitive impairment. So, perhaps, using more subtypes would produce different results. For example, cognitive impairment in complex cognitive processes would be more heterogeneous, and reaction time in simple cognitive tasks would still be homogenously impaired in the sample.

Another common problem in research that may cause divergence is how reaction time is measured and reported. Some studies have suggested that intraindividual variability is higher in individuals going through cognitive decline and, therefore, in patients with MCI[49],[51]. Although measurement of intraindividual variability itself can be worth investigating, it can create noise when the goal is to compare reaction time. Mean results from one or a few trials might not provide a good comparison measurement. CompCog does not have this problem since it uses the median reaction time derived from multiple trials. This would eliminate the variability problem that affects the MCI sample and does not affect the control sample. Even so, studying the intraindividual variability itself is another option for future studies with CompCog.

In addition to the abovementioned benefits of some computerized tests, two more can be added in the same context. First, simple choice reaction time can be evaluated longitudinally and without a learning effect. This enables longitudinal follow-up in which individuals will be compared with themselves in order to detect any decline right from its beginning, with the consequent possibility of early interventions.

Lastly, comparison between the reaction times for errors and correct responses in the two subtests that measure it (Survey test and Inhibitory Control test) showed that only the reaction times for correct responses could differentiate between the groups. Separated variables showing reaction times for errors and correct responses are not common in tests. The majority of computerized tests still use the same measurements used in paper-and-pencil tests, i.e. errors and total scores. The tests that investigate reaction time mainly focus on attention processes[52],[53], probably because the cognitive process construct is highly relatable to processing speed[23]. However, the results show that reaction times are not the same between situations of getting answers right or wrong. These differences might be worth considering as variables if new tests are created and might be worth investigating in future studies.

 

MissingLinkText

Regression models

The final model that best predicted MCI with the least number of variables included three reaction time measurements regarding attention and one regarding memory, which correctly classified 92.3% of the individuals. The direction of the reaction times in the inhibitory control test, the second task of the memory test and the second sequence of the implicit learning test differed from the direction of the fourth variable selected, i.e. the reaction time in the first task of the Stroop test. Upon closer inspection, we hypothesized that the MCI group committed more errors, while the healthy group took more time in order to avoid mistakes.

We propose that these results should be seen as an exploratory analysis. It could be difficult to use only the selected variables in a test, because variables inside tests from unrelated tasks were selected for the model. Even so, the model suggests that a reaction time score composed of performance levels in different tasks could have even higher accuracy than reaction time measured separately. This proposal has to go through further testing in future studies with a specific hypothesis and larger samples.

In conclusion, we can infer from the results that reaction time measurements through CompCog are an efficient and accurate way to screen for MCI. Although the initial cost of the equipment might be high, there is no maintenance cost for its administration thereafter. There is also the possibility of expanding the technology to other devices in future studies, such as to cellphones. Thus, this method could form a low-cost option for screening for MCI on a large scale. Low-cost options are especially necessary in low and middle-income countries[54]. It is not our proposal to use the test as a diagnostic tool but to bring in technology that allows doctors or caregivers to perform simple screening on individuals who are at the threshold of old age. Additional tests and investigations should be done to reach a diagnosis and indicate treatments, depending on the results.

In order to achieve the above objective, more evidence needs to be produced. To assess cognitive decline, it is important to compare individuals with themselves at different times[9], which is a matter that our study could not cover. The best way to screen for MCI would be to compare individuals' results year by year. Studies with follow-up could provide more evidence of the utility of CompCog for MCI screening.

Furthermore, two other variables that could have been controlled for were the individuals’ subjective cognitive decline and the time that elapsed between the first evaluation and the diagnosis. Controlling for the latter could have ensured that the length of time between the diagnosis and the neuropsychological assessment did not influence results. Controlling for subjective cognitive decline could have shown how and whether reaction time relates to cognitive complaints.

Lastly, the sample size can also be seen as a limitation of the present study. Although there is a need for larger samples to achieve more reliable results, there is a lack of studies exploring all MCI subtypes together. Most studies have explored Alzheimer's disease and amnestic MCI. Other MCI subtypes have been less investigated and, therefore, our findings remain relevant. Our results show that CompCog is a useful tool for screening for cognitive impairment regardless of the etiology, with reaction time measurements that are easy to obtain. CompCog can be a practical and advantageous instrument for selecting patients for a more comprehensive neuropsychological assessment and, therefore, enabling early diagnosis of MCI.

 

MissingLinkText

 

MissingLinkText

 

MissingLinkText

Conflict of interest:

There is no conflict of interest to declare.

Authors’ contributions:

LH, MM, HF: conceptualization; LH, MM, RS: data curation, investigation; LH, GB, HF: methodology; HF: validation and project administration; LH: formal analysis; LH, MM: writing-original draft; LH, GB, HF: writing-review & editing; HF: supervision.

Support

Larissa Hartle acknowledges funding from the Coordination for the Improvement of Higher Education Personnel (CAPES).

• References

• 1 Petersen RC, Stevens JC, Ganguli M, Tangalos EG, Cummings JL, DeKosky ST. Practice parameter: early detection of dementia: mild cognitive impairment (an evidence-based review). Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology 2001; 56 (09) 1133-1142 https://doi.org/10.1212/wnl.56.9.1133
• 2 Petersen RC, Doody R, Kurz A, Mohs RC, Morris JC, Rabins PV. et al. Current concepts in mild cognitive impairment. Arch Neurol 2001; 58 (12) 1985-1992 https://doi.org/10.1001/archneur.58.12.1985
• 3 Kasper S, Bancher C, Eckert A, Förstl H, Frölich L, Hort J. et al. Management of mild cognitive impairment (MCI): the need for national and international guidelines. World J Biol Psychiatry 2020; [phone] https://doi.org/10.1080/15622975.2019.1696473
• 4 Ritchie CW, Russ TC, Banerjee S, Barber B, Boaden A, Fox NC. et al. The Edinburgh Consensus: preparing for the advent of disease-modifying therapies for Alzheimer's disease. Alzheimers Res Ther 2017; 9 (01) 85 https://doi.org/10.1186/s13195-017-0312-4
• 5 Petersen RC. Mild cognitive impairment. Continuum (Minneap Minn) 2016; 22 (02) 404-418 https://doi.org/10.1212/CON.20210099202100990313
• 6 Breton A, Casey D, Arnaoutoglou NA. Cognitive tests for the detection of mild cognitive impairment (MCI), the prodromal stage of dementia: meta-analysis of diagnostic accuracy studies. Int J Geriatr Psychiatry 2019; 34 (02) 233-242 https://doi.org/10.1002/gps.5016
• 7 Tsoi KKF, Chan JYC, Hirai HW, Wong A, Mok VCT, Lam LCW. et al. Recall tests are effective to detect mild cognitive impairment: a systematic review and meta-analysis of 108 diagnostic studies. J Am Med Dir Assoc 2017; 18 (09) 807 https://doi.org/10.1016/j.jamda.2017.05.016
• 8 Giau VV, Bagyinszky E, An SSA. Potential fluid biomarkers for the diagnosis of mild cognitive impairment. Int J Mol Sci 2019; 20 (17) 4149 https://doi.org/10.3390/ijms20174149
• 9 Knopman DS, Petersen RC. Mild cognitive impairment and mild dementia: a clinical perspective. Mayo Clin Proc 2014; 89 (10) 1452-1459 https://doi.org/10.1016/j.mayocp.2014.06.019
• 10 Parra MA, Baez S, Allegri R, Nitrini R, Lopera F, Slachevsky A. et al. Dementia in Latin America: assessing the present and envisioning the future. Neurology 2018; 90 (05) 222-231 https://doi.org/10.1212/WNL.20210099202100994897
• 11 Oliveira MO, Brucki SMD. Computerized Neurocognitive Test (CNT) in mild cognitive impairment and Alzheimer’s disease. Dement Neuropsychol 2014; 8 (02) 112-116 https://doi.org/10.1590/S1980-57642014DN82000005
• 12 Razak MAA, Ahmad NA, Chan YY, Kasim NM, Yusof M, Ghani MKAA. et al. Validity of screening tools for dementia and mild cognitive impairment among the elderly in primary health care: a systematic review. Public Health 2019; 169: 84-92 https://doi.org/10.1016/j.puhe.2019.01.001
• 13 Salthouse TA. Aging and measures of processing speed. Biol Psychol 2000; 54 1-3 35-54 https://doi.org/10.1016/s0301-0511(00)00052-1
• 14 Booth T, Dykiert D, Corley J, Gow AJ, Morris Z, Maniega SM. et al. Reaction time variability and brain white matter integrity. Neuropsychology 2019; 33 (05) 642-657 https://doi.org/10.1037/neu0000483
• 15 Phillips M, Rogers P, Haworth J, Bayer A, Tales A. Intra-individual reaction time variability in mild cognitive impairment and Alzheimer's disease: gender, processing load and speed factors. PLoS One 2013; 8 (06) e65712 https://doi.org/10.1371/journal.pone.0065712
• 16 Hong Y, Alvarado RL, Jog A, Greve DN, Salat DH. Serial reaction time task performance in older adults with neuropsychologically defined mild cognitive impairment. J Alzheimers Dis 2020; 74 (02) 491-500 https://doi.org/10.3233/JAD-191323
• 17 Salthouse TA. The processing-speed theory of adult age differences in cognition. Psychol Rev 1996; 103 (03) 403-428 https://doi.org/10.1037/0033-295x.103.3.403
• 18 Martorelli M, Hartle L, Coutinho G, Mograbi DC, Chaves D, Silberman C. et al. Diagnostic accuracy of early cognitive indicators in mild cognitive impairment. Dement Neuropsychol 2020; 14 (04) 358-365 https://doi.org/10.1590/1980-57642020dn14-040005
• 19 Haworth J, Phillips M, Newson M, Rogers PJ, Torrens-Burton A, Tales A. Measuring information processing speed in mild cognitive impairment: clinical versus research dichotomy. J Alzheimers Dis 2016; 51 (01) 263-275 https://doi.org/10.3233/JAD-150791
• 20 Chen K-C, Weng C-Y, Hsiao S, Tsao W-L, Koo M. Cognitive decline and slower reaction time in elderly individuals with mild cognitive impairment. Psychogeriatrics 2017; 17 (06) 364-370 https://doi.org/10.1111/psyg.12247
• 21 Andriuta D, Diouf M, Roussel M, Godefroy O. Is reaction time slowing an early sign of Alzheimer’s disease? A meta-analysis. Dement Geriatr Cogn Disord 2019; 47 4-6 281-288 https://doi.org/10.1159/000500348
• 22 Bailon O, Roussel M, Boucart M, Krystkowiak P, Godefroy O. Psychomotor slowing in mild cognitive impairment, Alzheimer's disease and lewy body dementia: mechanisms and diagnostic value. Dement Geriatr Cogn Disord 2010; 29 (05) 388-396 https://doi.org/10.1159/000305095
• 23 Fernaeus S-E, Östberg P, Wahlund L-O. Late reaction times identify MCI. Scand J Psychol 2013; 54 (04) 283-285 https://doi.org/10.1111/sjop.12053
• 24 Okonkwo OC, Wadley VG, Griffith HR, Ball K, Marson DC. Cognitive correlates of financial abilities in mild cognitive impairment. J Am Geriatr Soc 2006; 54 (11) 1745-1750 https://doi.org/10.1111/j.1532-5415.2006.00916.x
• 25 Giovannetti T, Bettcher BM, Brennan L, Libon DJ, Kessler RK, Duey K. Coffee with jelly or unbuttered toast: commissions and omissions are dissociable aspects of everyday action impairment in Alzheimer's disease. Neuropsychology 2008; 22 (02) 235-245 https://doi.org/10.1037/0894-4105.22.2.235
• 26 Miller JB, Barr WB. The technology crisis in neuropsychology. Arch Clin Neuropsychol 2017; 32 (05) 541-554 https://doi.org/10.1093/arclin/acx050
• 27 Ferreira D, Correia R, Nieto A, Machado A, Molina Y, Barroso J. Cognitive decline before the age of 50 can be detected with sensitive cognitive measures. Psicothema 2015; 27 (03) 216-222 https://doi.org/10.7334/psicothema2014.192
• 28 Levinson D, Reeves D, Watson J, Harrison M. Automated neuropsychological assessment metrics (ANAM) measures of cognitive effects of Alzheimer's disease. Arch Clin Neuropsychol 2005; 20 (03) 403-408 https://doi.org/10.1016/j.acn.2004.09.001
• 29 Charchat H, Nitrini R, Caramelli P, Sameshima K. Investigação de marcadores clínicos dos estágios iniciais da doença de Alzheimer com testes neuropsicológicos computadorizados. Psicol Reflex Crit 2001; 14 (02) 305-316 https://doi.org/10.1590/S0102-79722001000200006
• 30 Fichman HC, Nitrini R, Caramelli P, Sameshima K. A new Brief computerized cognitive screening battery (CompCogs) for early diagnosis of Alzheimer's disease. Dement Neuropsychol 2008; 2 (01) 13-19 https://doi.org/10.1590/S1980-57642009DN20100004
• 31 Araujo VC, Lima CMB, Barbosa ENB, Furtado FP, Charchat-Fichman H. Impact of age and schooling on performance on the Brief Cognitive Screening Battery: a study of elderly residents in the City of Rio de Janeiro, Brazil. Psychol Neurosci 2018; 11 (03) 317-328 https://doi.org/10.1037/pne0000118
• 32 Brucki SMD, Nitrin R, Caramelli P, Bertolucci PHF, Okamoto IH. Sugestões para o uso do mini-exame do estado mental no Brasil. Arq Neuropsiquiatr 2003; 61(3B) 777-781 https://doi.org/10.1590/s0004-282x2003000500014
• 33 Nitrini R, Lefèvre BH, Mathias SC, Caramelli P, Carrilho PE, Sauaia N. et al. Testes neuropsicológicos de aplicação simples para o diagnóstico de demência. Arq Neuropsiquiatr 1994; 52 (04) 457-465 https://doi.org/10.1590/s0004-282x1994000400001
• 34 Veras C, Hartle L, Araújo VC, Charchat-Fichaman H. Estudo normativo da Escala de Depressão Geriátrica em amostra de idosos do Rio de Janeiro. Rev Neuropsicol Latinoam 2020; 12 (03) 41-50 https://doi.org/10.5579/rnl.2016.0561
• 35 Sanchez MAS, Correa PCR, Lourenço RA. Cross-cultural adaptation of the "Functional Activities Questionnaire - FAQ" for use in Brazil. Dement Neuropsychol 2011; 5 (04) 322-327 https://doi.org/10.1590/S1980-57642011DN05040010
• 36 Santos RL, Virtuoso Júnior JS. Confiabilidade da versão brasileira da escala de atividades instrumentais da vida diária. Rev Bras Promoç Saúde 2012; 21 (04) 290-296 https://doi.org/10.5020/575
• 37 Naidel E, Faria CDA, Veiga H, Alves D, Lima DA, Novaes R. et al. Perfis neuropsicológicos do Comprometimento Cognitivo Leve no envelhecimento (CCL). Rev Neuropsicol Latinoam 2015; 7: 15-23 https://doi.org/10.5579/rnl.2015.0257
• 38 Stroop JR. Studies of interference in serial verbal reactions. J Exp Psychol 1935; 18 (06) 643-662 https://doi.org/10.1037/h0054651
• 39 Hartle L, Mendes-Santos L, Barbosa E, Balboni G, Charchat-Fichman H. Evidence of validity of a novel version of the computerized cognitive screening battery CompCog. Dement Neuropsychol 2021; 15 (04) 485-496 https://doi.org/10.1590/1980-57642021dn15-040010
• 40 Monsch AU, Bondi MW, Butters N, Paulsen JS, Salmon DP, Brugger P. et al. A comparison of category and letter fluency in Alzheimer's disease and Huntington's disease. Neuropsychology 1994; 8 (01) 25-30 https://doi.org/10.1037/0894-4105.8.1.25
• 41 Joubert S, Gardy L, Didic M, Rouleau I, Barbeau EJ. A meta-analysis of semantic memory in mild cognitive impairment. Neuropsychol Rev 2021; 31 (02) 221-232 https://doi.org/10.1007/s11065-020-09453-5
• 42 Kit KA, Mateer CA, Tuokko HA, Spencer-Rodgers J. Influence of negative stereotypes and beliefs on neuropsychological test performance in a traumatic brain injury population. J Int Neuropsychol Soc 2014; 20 (02) 157-167 https://doi.org/10.1017/S1355617713001264
• 43 Brailean A, Steptoe A, Batty GD, Zaninotto P, Llewellyn DJ. Are subjective memory complaints indicative of objective cognitive decline or depressive symptoms? Findings from the english longitudinal study of ageing. J Psychiatr Res 2019; 110: 143-151 https://doi.org/10.1016/j.jpsychires.2018.12.005
• 44 Mitchell AJ, Beaumont H, Ferguson D, Yadegarfar M, Stubbs B. Risk of dementia and mild cognitive impairment in older people with subjective memory complaints: meta-analysis. Acta Psychiatr Scand 2014; 130 (06) 439-451 https://doi.org/10.1111/acps.12336
• 45 Jorm AF, Butterworth P, Anstey KJ, Christensen H, Easteal S, Maller J. et al. Memory complaints in a community sample aged 60-64 years: associations with cognitive functioning, psychiatric symptoms, medical conditions, APOE genotype, hippocampus and amygdala volumes, and white-matter hyperintensities. Psychol Med 2004; 34 (08) 1495-1506 https://doi.org/10.1017/s0033291704003162
• 46 Park J-H, Jung M, Kim J, Park HY, Kim J-R, Park J-H. Validity of a novel computerized screening test system for mild cognitive impairment. Int Psychogeriatr 2018; 30 (10) 1455-1463 https://doi.org/10.1017/S1041610218000923
• 47 Memória CM, Yassuda MS, Nakano EY, Forlenza OV. Contributions of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI) for the diagnosis of MCI in Brazil. Int Psychogeriatr 2014; 26 (09) 1483-1491 https://doi.org/10.1017/S1041610214000726
• 48 Tornatore JB, Hill E, Laboff JA, McGann ME. Self-administered screening for mild cognitive impairment: initial validation of a computerized test battery. J Neuropsychiatry Clin Neurosci 2005; 17 (01) 98-105 https://doi.org/10.1176/jnp.17.1.98
• 49 Gorus E, De Raedt R, Lambert M, Lemper J-C, Mets T. Reaction times and performance variability in normal aging, mild cognitive impairment, and Alzheimer's disease. J Geriatr Psychiatry Neurol 2008; 21 (03) 204-218 https://doi.org/10.1177/0891988708320973
• 50 de Jager CA, Schrijnemaekers A-CMC, Honey TEM, Budge MM. Detection of MCI in the clinic: evaluation of the sensitivity and specificity of a computerised test battery, the Hopkins Verbal Learning Test and the MMSE. Age Ageing 2009; 38 (04) 455-460 https://doi.org/10.1093/ageing/afp068
• 51 Anstey KJ, Mack HA, Christensen H, Li S-C, Reglade-Meslin C, Maller J. et al. Corpus callosum size, reaction time speed and variability in mild cognitive disorders and in a normative sample. Neuropsychologia 2007; 45 (08) 1911-1920 https://doi.org/10.1016/j.neuropsychologia.2006.11.020
• 52 Park J-H, Park J-H. Development of computerized screening test items for mild cognitive impairment. Ther Sci Rehabil 2018; 7 (04) 31-41 https://doi.org/10.22683/tsnr.2018.7.4.031
• 53 Schweiger A, Doniger GM, Dwolatzky T, Jaffe D, Simon ES. Reliability of a novel computerized neuropsychological battery for mild cognitive impairment. Neuropsychilogica 2003; 1: 407-413
• 54 Hartle L, Charchat-Fichman H. Mild cognitive impairment history and current procedures in low- and middle-income countries: a brief review. Dement Neuropsychol 2021; 15 (02) 155-163 https://doi.org/10.1590/1980-57642021dn15-020001

Publikationsverlauf

Eingereicht: 18. März 2021

Angenommen: 31. August 2021

Artikel online veröffentlicht:
06. Februar 2023