neuropsychology - attention - psychometrics - cognition - stroop test
neuropsicologia - atenção - psicometria - cognição - teste de stroop
Despite of intense development in the last decades neuropsychologists still have a
limited number of adapted, validated and standardized neuropsychological tests to
use in clinical and research contexts where patients have low formal education[1 ]. An important aspect to be evaluated by neuropsychologists are the components of
the attentional system, which involves automatic and controlled processes. The attentional
processes influence different ecological contexts, such as reading, writing, arithmetic
and are strongly related to executive functions, a set of cognitive skills responsible
for controlling and handling goal-oriented behavior[2 ]. Tests evaluating attention are generally used in research and clinical contexts,
yet the examinee’s sociodemographic and educational background constantly represents
an important barrier for test selection, application and interpretation[3 ]. Recently, a new set of attention’s tests seems to overcome some of these limitations
in our context, but some aspects related to its properties must be investigated[4 ],[5 ].
Petersen and Posner[2 ] suggested a human attention system subdivided into three functionally independent
networks. The alerting network allows maintenance of a vigilant and alert state, the
orienting network is responsible for the movement of attention through space to attend
to sensory events, and the executive control network allows for the monitoring and
resolution of conflict between expectation, stimulus, and response. The three attentional
networks have a modular nature, yet strongly interconnected[6 ]. The automatic processes (alerting and orienting) have great conceptual overlap
with the definition of processing speed, a cognitive property associated with the
time spent in different tasks execution frequently found in psychometric studies[7 ]. The controlled ones are superposed with components of the executive function, particularly
the attentional components of the inhibitory control system[8 ].
One of the most used methods to evaluate attentional processes is the interference
paradigm, where an automatic attentional routine must be inhibited favoring a controlled
response which occurs in the classic Stroop Color-Word Test (SCWT)[9 ]. However, the interference effect of the SCWT could be inaccurate for specific populations
such as preschoolers, illiterates or even individuals with low formal schooling since
reading efficiency moderates the time of the task execution[10 ]. The Five Digit Test (FDT) was design to overcome some of these limitations requiring
to test-takers to read or count Arabic numerals (1–5) or count stars[11 ]. Four parts compose the FDT: reading, counting, choosing (selective attention),
and switching (alternate attention)[11 ]. The Oral Trails Test (OTT)[12 ] is another alternative to the SCWT combined with elements from the Trail Making
Test[13 ]. The OTT interference effect is expected from shapes and colors of four fruits:
banana (yellow), apple (red), grape (purple) and orange (orange). Additionally to
the interference effect, the OTT has a visual search component and demands the knowledge
of the ascending numerical order of Arabic digits from 1-to-20. Five parts compose
the OTT: semantic search, spatial search, selective attention, and interchange (alternating
attention). The OTT also presents low literacy requirements.
Beyond its psychometric properties, the practical application of a test is extremely
important in Neuropsychology. An important and often neglected aspect of neuropsychological
testing is the ecological validity. This property refer to which extent the score
on a neuropsychological test relates to patient’s behavior in everyday life[14 ]. Although most of the neuropsychological tests may be useful for of the clinical
profile in psychiatric or neurological disorders, several tests have limited potential
to predict ecological outcomes[15 ]. There are several evidences of the attentional system importance on everyday learning
and behavior[2 ], but the extension of the association of specific tests with everyday outcomes is
of extreme relevance for the validation of new tests measuring automatic and controlled
processes.
The Cognitive Failures Questionnaire (CFQ)[16 ] is a reliable measurement of self-reported problems in perceptual, memorial and
motoric functions in daily life and have an association with attentional performance[17 ]. Considering the importance of the proximity of an objective cognitive test with
real-world situations (ecological validity) for its practical implications, we sought
to investigate the association between the FDT and the OTT measures (automatic and
controlled processes) with self-reported cognitive failures as measured by the CFQ.
METHODS
Participants
Sixty Brazilian adults with ages between 18 and 65 years old were enrolled in this
study. The inclusion criteria was age over 18 years, volunteer interest in participation
and written consent for participation. The exclusion criteria was severe sensory or
motors deficits, which could affect participants’ performance in the proposed tasks,
history of psychotropic substances abuse or related history of neurological disease
and/or severe psychiatry disorders, which could affect the global cognitive status
(e.g. stroke, epilepsy, multiple learning disorders, developmental disorders, uncontrolled
systemic diseases). This study was approved by the Faculdade de Ciências Médicas de
Minas Gerais ethics committee board (process 745.264) and is in accordance to Helsiki
declaration. The subjects gave written consent for participation.
Neuropsychological and psychological assessment
Five Digit Test (FDT)
11
: The FDT is a numeric-Stroop paradigm applying four steps. Parts 1 and 2 involve automatic
processes such as reading (the Arabic algorisms 1,2,3,4 and 5) and counting (quantities from one-to-five). The Part 3 (choosing) involves interference control since an automatic numerical transcoding (i.e., naming - transform a number from the Arabic digital format to the oral verbal format) has to
be inhibited in favor of a controlled one (i.e., count Arabic digits that not represent
the set cardinality) (e.g., stimulus =”1,1,1” and response =”three”). Part 4 (shifting ) involves a set-shift from rules of Part 1 to Part 3 and vice-versa depending on
an explicit marker. Executive scores are calculates for inhibition (choosing – reading) and flexibility (shifting – reading). The used measure to evaluate participants’ performance was
the time spent to complete the tasks in each part. Faster times indicates a better
performance. An example of the test stimuli is show in [Figure 1 ].
Figure 1 Five Digit Test. Parts 1 and 2 (Reading and Counting) are related to simple processing
speed and automatic attentional processes. Parts 3 and 4 (Choosing and Shifting) involve
inhibition and cognitive flexibility and are related to executive functions and controlled
attentional processes. Inhibition and Flexibility scores are computed subtracting
the Reading time from Choosing and Shifting times, respectively. A disproportionate
slowness in the Choosing/Inhibition and Shifting/Flexibility when compared to normative
parameters may be indicative of deficits in executive functions.
Oral Trails Test (OTT)
12
: The test has 5 Parts. In Part 1 (naming ), the participant is required to name the number and the fruit in each one of a series
of 20 squares printed in one page. All stimulus are shown in a canonical way in the
first step. In Part 2 (spatial search ), the stimulus are shown in a random numerical order and participants are asked to
name the number and the fruit in each one of the 20 squares, but in its ascending
numerical order. Parts 3 (choosing ) and 4 (executive search ) consist in naming the number and the fruit that have the color (but not the shape)
of the stimulus in each square with the stimulus in Part 4 disposed in a random order.
In the Part 5 (shifting ), participants are instructed to perform the same as Part 3 (no visual search required),
but depending on an explicit marker they had to change the rule and name the shape
(not the color) of the fruit of the stimulus. The used measure to evaluate participants’
performance was the time spent to complete the task in each part. Faster times indicates
a better performance. An example of the test stimuli is show in [Figure 2 ].
Figure 2 Oral Trails Test. Part 1 (Naming) involves the naming of ordered stimuli. Part 2 (spatial
search) involves naming and visual search of non-ordered stimuli. These two steps
are related do simple processing speed and attention. Parts 3 (choosing), 4 (executive
search), and 5 (shifting) involve processes related to inhibition and cognitive flexibility,
which are measures of controlled attentional processes and executive functions. A
disproportionate slowness in time execution of choosing and shifting when compared
to reading, and of executive search when compared to visual search, based on normative
parameters, may be indicative of deficits in executive functions.
Cognitive Failures Questionnaire (CFQ)
16
: CFQ is an instrument of self-reported everyday cognitive failures frequency. The CFQ
consists of 25 items. Participants are asked to report the frequency of incidents
such as ‘Do you read something and find you haven’t been thinking about it and must
read it again?’ in the past 6 months on a 5-point Likert-type scale. Following the
recommendations of Rast and colleagues[18 ], the CFQ was split into three factors: Forgetfulness, Distractibility, and False
Triggering, in addition to the total score. The first factor is associated with memory
complaints, the second with inattentive behavior, and the last one may be seen as
resulting from loss of activation in attentional resources. Higher scores are representative
of more frequent cognitive failures on daily life. The translation and preliminary
study of validity was sent for appreciation to a scientific journal.
Statistical analysis
After data distribution assessment and descriptive statistics the association between
the cognitive measures and the CFQ was investigated through partial correlations controlling
the effect of age (a factor previously associated with cognitive performance[7 ],[8 ] and cognitive failures[18 ]). The prior analysis had an exploratory nature and aimed to identify candidate variables
for the following analysis. Measures associated to CFQ at p < 0.10 were added in stepwise
regression models containing CFQ as the dependent measure and the associated variables
as predictors. The stepwise strategy was adopted to avoid colinearity. The significance
value for regression models was established at p < 0.05. Procedures were performed
in the SPSS 20.0.
RESULTS
[Table 1 ] presents the demographic profile of the sample. Participants were predominantly
young adults with ages between 18 and 26 years (67%). [Table 2 ] presents the participants’ scores (mean and standard deviation) in the neuropsychological
and psychological measures.
Table 1
Demographic characteristics of the sample (n = 60).
Age
18 – 23
34%
23 – 26
33%
27+
33%
Gender
Male
32%
Female
68%
Highest academic degree
Primary
5%
Secondary/technical
58%
College
37%
Mental disorder or neurological disease
No
80%
Yes
20%
Use of medication
No
73%
Yes
27%
Socioeconomic class
A
22%
B1
15%
B2
32%
C1
13%
C2
15%
D-E
3%
Occupation
Studying
32%
Studying and working
30%
Not studying nor working
6%
Working
32%
Table 2
Participants’ scores in the Cognitive Failures Questionnaire and performance in the
attentional tests: FDT and OTT.
Mean
SD
Cognitive failures
44.35
14.91
Cognitive failures (forgetfulness)
15.68
5.03
Cognitive failures (distractibility)
14.63
5.25
Cognitive failures (false triggering)
12.37
5.43
FDT – Reading (time)
22.67
6.62
FDT - Counting (time)
25.43
5.77
FDT – Inhibition (time)
15.97
7.57
FDT – Flexibility (time)
24.82
10.12
OTT – Naming (time)
20.08
4.39
OTT – Visual search (time)
45.52
10.8
OTT – Choosing (time)
29.77
10.43
OTT – Executive search (time)
54.08
13.83
OTT – Shifting (time)
34.65
11.40
FDT: Five Digit Test; OTT: Oral Trails Test; SD: standard deviation.
[Table 3 ] presents the coefficients of the partial correlation (controlled by age) between
the attentional scores measured by the FDT and the OTT and the CFQ factors. Statistical
significant correlations between the FDT and the CFQ occurred more consistently in
the flexibility measure (step 4) with moderate correlations with CFQ forgetfulness, False Triggering and total
score. For the OTT measures, we observed the same pattern with significant correlations
between the measures of controlled attention (choosing, executive-search and shifting)
and the CFQ factors, except for the association of inhibition with cognitive failures
by false alarm. This preliminary analysis showed the controlled/executive attentional
processes to be more related to cognitive failures than the automatic attentional
process in this sample.
Table 3
Coefficients of partial correlation (controlled by age) between the FDT and OTT scores
and the Cognitive Failures Questionnaire scores.
Cognitive tests
Cognitive failures questionnaire
Total
Forgetfulness
Distractibility
False triggering
FDT – Reading (time)
0.161
0.138
0.047
0.231*
FDT - Counting (time)
0.238*
0.194
0.175
0.232*
FDT – Inhibition (time)
0.180
0.254*
0.173
0.099
FDT – Flexibility (time)
0.314**
0.352***
0.196
0.302**
OTT – Naming (time)
0.153
0.122
0.207
0.100
OTT – Visual search (time)
0.000
0.038
-0.049
0.008
OTT – Choosing (time)
0.233*
0.219*
0.263**
0.177
OTT – Executive search (time)
0.265**
0.231*
0.218*
0.263**
OTT – Shifting (time)
0.312**
0.249*
0.343***
0.261**
FDT: Five Digit Test; OTT: Oral Trails Test. *p < 0.1, **p < 0.05, ***p < 0.01.
In [Table 4 ] is presented data from the stepwise multiple linear regression model, analyzing
the influence of age and the cognitive variables on CFQ factors. Age and FDT flexibility
accounted for by 26% of the CFQ total score variance, and for by 19% of the False
Triggering factor variance. Age and OTT shifting represented 18% of the variance on
cognitive failures by forgetfulness and 25% of the variance on cognitive failures
by Distractibility. Results suggest that the cognitive flexibility measured both for
the FDT and the OTT was a significant predictor of self-reported cognitive failures
in general.
Table 4
Stepwise multiple linear regression model analyzing the contribution of age and neuropsychological
tests for cognitive failures.
Outcome
F
p
R2
Predictors
Beta
P
CFQ total score
10.25
< 0.001
26%
Age
0.543
< 0.001
FDT – flexibility
0.305
0.016
CFQ forgetfulness
7.66
0.001
18%
Age
0.451
< 0.001
OTT – shifting
0.359
0.006
CFQ distractibility
10.03
< 0.001
25%
Age
0.679
< 0.001
OTT – shifting
0.404
0.008
CFQ false triggering
7.78
0.001
19%
Age
0.481
< 0.001
FDT – flexibility
0.303
0.020
FDT: Five Digit Test; OTT: Oral Trails Test; CFQ: Cognitive Failures Questionnaire.
DISCUSSION
In this study, controlled aspects of the cognitive attentional system, measured by
the Stroop paradigms FDT and OTT, were associated with the frequency of cognitive
failures in everyday life. The cognitive flexibility of both tests explained a medium
portion of the variance in most of the measures of the CFQ, even controlling for the
age effect. Weaker correlations were observed between the FDT flexibility measure,
the Distractibility factor of the CFQ and between the OTT shifting, and the Forgetfulness
and False Triggering factors of the CFQ.
Bradbent and Colleagues[16 ] explain that cognitive failures differ from failures on capacity, which means that
cognitive failures are a lapse and a momentary change on normal flux of certain function.
However, significant correlations are reported between cognitive failures and performance
on ecological measures of attention and memory[19 ]. Then, cognitive traits may influence cognitive failures in everyday life, yet those
cognitive failures not always will be a reflection of the cognitive performance since
they are transient, as seen in other studies[20 ],[21 ]. This is in accordance with our results, where moderate correlations between the
flexibility component and the cognitive failures would reflect an association and
not a perfect superposition of this cognitive domain with a more complex behavioral
concept. The lack of correlation between the FDT flexibility and the Distractibility
factor was unexpected since attentional resources would influence this kind of cognitive
failures. It could be possible that transient factors such as the level of daily demands
are more relevant for cognitive failures related to Distractibility. For example,
people after a significant change in life style, as retirement, may report less distracted
behavior[22 ],[23 ].
Most important for our purposes, measures of the FDT and the OTT were predictive of
real-world problems such as cognitive failures in everyday activities/situations.
Human capacity to selectively attend, choose something and focusing on that or suppressing
attention of a stimuli is enabled for by attentional control and is driven by the
intrinsic properties of a stimuli[2 ]. Then, the design of a test is of utmost importance for an interference control
effect (attentional inhibition). The FDT and OTT, however, seems to go beyond interference
control demanding from subjects to perform a dual-task version of the previous parameters
of the tests. The flexibility component of both tests was the most predictive of cognitive
failures. It has been described that cognitive flexibility is a more complex executive
function building on inhibitory and working memory demands[8 ]. Cognitive flexibility permit us to adjust performance to changed demands. The impact
of these executive functions increase with task complexity and daily life demands[8 ] which help us to understand why lower performance on FDT and OTT was associated
with higher levels of cognitive failures. Cognitive failures are expected to increase
in a context of ascending amount of information, which intensify the necessity for
the identification of predictors of errors that occurs due to slips of memory and
lapses of attention. People with high CFQ scores usually bump into things, forget
names or lose task goals during task execution and so are more likely to fail during
the pursuit of task goals or even to suffer more accidents[24 ]. Then, the assessment of attentional control and other executive measures could
be helpful in contexts of high daily demands with increased risk for errors due to
cognitive failures.
This work is limited by its power of generalization since we investigated only healthy
adults and only one ecological measure (i.e., cognitive failures) was used as an associated
outcome. Further studies should investigate the validity of the FDT and OTT in a larger
sample. The effect of FDT and OTT measures on outcomes such as cognitive failures
should also be investigated in clinical populations since alterations in the attentional-executive
system if often present[25 ].
Our results suggest that FDT and OTT showed evidences of ecological validity, using
as parameter everyday cognitive failures reported by the participants. Ecological
validity is an important aspect to be verified on neuropsychological tests, since
in many situations they are used to evaluate prognosis or to predict patients’ behavior.
