Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS): discrete and regression-based norms for the Brazilian context

ABSTRACT

Background: The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) has been recently developed as a brief, practical, and feasible tool for cognitive impairment in multiple sclerosis (MS). Objective: This study aimed to provide continuous and discrete normative values for the BICAMS in the Brazilian context. Methods: Normatization was achieved using six hundred and one healthy controls from the community assessed at five Brazilian geopolitical regions. Results: Mean raw scores, T scores, percentiles, and Z scores for each BICAMS measure are provided, stratified by age and educational level. Regression-based norms were provided by converting raw scores to scaled scores, which were regressed on age, gender, and education, yielding equations that can be used to calculate the predicted scores. Regression analyses revealed that age, gender, and education significantly influenced test results, as in previous studies. Conclusions: The normative data of the BICAMS to the Brazilian context presented good representativeness, improving its use in daily clinical practice.

RESUMO

Antecedentes: O BICAMS foi desenvolvido como uma ferramenta breve, prática e confiável para avaliar o comprometimento cognitivo na esclerose múltipla (EM). Objetivo: Neste estudo, objetivamos fornecer dados normativos para o BICAMS. Métodos: Normatização foi realizada com seiscentos e um controles saudáveis da comunidade avaliados das cinco regiões geopolíticas brasileiras. Resultados: Escores brutos médios, escore T, percentil e escore Z para cada medida do BICAMS são fornecidos e estratificados por idade e nível educacional. Normas baseadas em regressão foram obtidas através da conversão dos pontos brutos em pontos ponderados, produzindo parâmetros de regressão que podem ser usados para calcular os escores preditos. As análises de regressão revelaram que idade, gênero e educação influenciaram significativamente nos resultados do teste, assim como em estudos prévios. Conclusão: Normas do BICAMS para o contexto brasileiro apresentaram boa representatividade, contribuindo para a utilização na prática clínica diária.

Keywords:

Multiple Sclerosis - Cognitive Dysfunction - Neuropsychology

Palavras-chave:

Esclerose Múltipla - Disfunção Cognitiva - Neuropsicologia

INTRODUCTION

Over 40% of multiple sclerosis (MS) patients can present cognitive impairment (CI)[1] ^- [4]. CI often manifests in decreased processing speed, learning and memory deficits, and less frequently in executive dysfunction[2],[3]. Cognitively disabled patients are more likely to be unemployed and report fewer extra-curricular and social activities[5],[6]. Appropriate test measures for the identification of CI are essential for the clinical management of the disease. Ideally, all MS patients would be routinely evaluated and/or monitored for CI, with similar measures being employed across specialized care centers.

An extensive neuropsychological assessment can be costly and time-consuming, with standards for the cognitive evaluation varying between providers. The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) is a collection of tests chosen by an international panel to standardize and facilitate routine monitoring of cognition in MS patients[7]. Transcultural studies between BICAMS from Brazil and the United States (US) have demonstrated its reliability and validity[8].

The present study was conducted to establish the discrete and regression-based norms of the BICAMS to the Brazilian context.

METHODS

Participants

Six hundred and one healthy control volunteers were recruited. The healthy controls were from the five Brazilian regions (north, northeast, south, southeast, and central-west), as recommended by the Brazilian Institute of Geographic and Statistics[9]. The Ethics Committee of the Medical School of Ribeirão Preto, University of São Paulo, approved all research procedures for this study.

Quality control of data acquisition

Qualification procedures were required from all the examiners. Psychologists and health professionals participated in an interactive training on administration and scoring procedures of the BICAMS. The protocol was administered in a standardized manner following commonly used manual-based instructions[10]. The two sub-scales of the Hospital Anxiety and Depression Scale (HADS)[11] were also administered as markers of major depression and generalized anxiety disorder. The Mini-Mental State Examination test (MMSE)[12] was administered for screening and to exclude cognitive impairment and possible dementia. No healthy controls were excluded based on their HADS or MMSE performance.

BICAMS standardization and scoring procedures

In brief, the Symbol Digit Modalities Test (SDMT)[13] assesses complex scanning, visual tracking[14], processing speed, and attention. The SDMT oral version was employed in this study, following previous validation studies[4],[15] and consensus opinion papers[16]. The task consists of a series of symbols presented at random, each with a blank space below, and participants are asked to name the number that matches each symbol. The maximum duration of the test is 90 seconds for the participant to complete each assessment (oral and written). When both forms of the test are administered, it is recommended that the written version be given first. It takes about five minutes to complete the test. The number of correct substitutions within the 90-second interval is recorded, with 110 being the maximum score for each form (written and oral). The translated standardized instructions of the SDMT for the Brazilian context are provided in the Technical and Interpretative Manual.

The California Verbal Learning Test - Second Edition (CVLT-2)[17] is a commonly used test of auditory/verbal learning and memory[18],[19]. The total CVLT-2 raw score is obtained by the sum of the correct words said in the five learning trials (T1 to T5). The total raw score for the learning trial obtained for the CVLT-2 is 80 points. The translated standardized instructions of the CVLT-2 to the Brazilian context are: “Eu vou ler uma lista de palavras para você. Preste atenção porque quando eu terminar de ler eu quero que você me diga o máximo de palavras que puder. Você pode repeti-las em qualquer ordem me dizendo o máximo de palavras que você se lembrar. Você está pronto? Leia a lista (A) de palavras em um mesmo ritmo, utilizando um pouco mais do que 1 segundo de intervalo entre as palavras. Dessa forma, a lista completa deve levar entre 18 e 20 segundos. E então diga: Pode começar .”

Instructions for the trials 2 to 5 (T2 to T5) in Brazilian Portuguese are: “Eu vou ler para você novamente a mesma lista de palavras. Como da primeira vez, eu quero que você me diga o máximo de palavras que puder se lembrar e em qualquer ordem. Procure repetir as mesmas palavras que você me falou na primeira vez”.

The Brief Visual Memory Test-Revised (BVMT-R)[20] is a visual/spatial memory test wherein subjects view a stimulus card with six figures for 10 seconds, and then render the figures from memory. There are three learning trials, which are summed to produce the final BVMT-R score (maximum score 36). The scores reflect visuospatial memory. The translated standardized instructions of the BVMT-R to the Brazilian context are: “Agora eu vou mostrar para você uma página com alguns desenhos. Eu preciso que você olhe atentamente para cada um deles durante 10 segundos. Procure prestar atenção no formato e na posição deles na folha. Eu vou te mostrar uma vez, daí retiro os desenhos e você os fará de memória aqui nesta folha (mostrar uma folha de sulfite A4 em branco). Nós faremos um treino, ou seja, vou te mostrar os desenhos e você irá desenhá-los por três vezes para você poder aprender e memorizar os desenhos.”

Instructions for the trials 2 and 3 (T2 and T3) in Brazilian Portuguese are: “Muito bem, agora vou mostrar mais uma vez porque quero ver se você aprendeu mais alguma figura”. Present the page with the stimuli for 10 seconds, then remove and deliver the blank sheet.

Statistical analysis

Statistical analysis was performed with the Statistical Package for the Social Sciences (v21.0, SPSS Inc, Chicago). Normality tests for each variable were performed with the Shapiro-Wilk test.

To the regression-based norms, the same methodology employed previously[21],[22] were applied in the present study. Each raw value was rank-ordered, converted to a proportion estimated, and then multiplied by 100 to derive percentile ranks. The ranks were then converted to scaled scores using the same conversion for all variables (mean=10, standard deviation=3). Thus, these steps allowed to convert the raw scores to a scaled score metrics based on a cumulative frequency distribution. The new variables are the Scaled Scores (SS) that were carried forward to the next step.

The SSs were inspected to ensure if the data were normally distributed. The scaled scores were then entered one at a time, as the dependent variables in multiple regression analyses used to model the proportion of variance in performance accounted for by the block entry of four demographic variables: age, age², gender, and years of education. Age and age² were coded in years. The b weights for each demographic variable, a predictive constant, and the standard deviation of the group’s raw residuals were derived for each test.

RESULTS

Demographics of the normative sample are shown in [Table 1]. The sample consisted of 65.6% (n=394) females, and age between 18 to ≥ 70 years. Most participants were between 30 and 39 years old (30.1%, n=181) and had a high educational level (59.6%, n=358, >12 years). The MMSE and HADS scores were classified as normal.

Table 1 Demographic characteristics of the normative sample (n=601).

Demographic data		Percentage (freq.)	Mean±SD (95%CI)
Gender	Female	65.6 (394)	-
	Male	34.4 (207)	-
Region	Central West	32.1 (193)	-
	Northeast	6.5 (39)	-
	South	35.3 (212)	-
	Southeast	26.1 (157)	-
Age groups (years)	18 to 29	22.3 (134)	-
	30 to 39	30.1 (181)	-
	40 to 49	17 (102)	-
	50 to 59	17.5 (105)	-
	60 to 69	53 (8.8)	-
	≥70	26 (4.3)	-
Classes of education (years)	1 to 8	10.8 (65)	-
	9 to 11	29.6 (178)	-
	>12	59.6 (358)	-
MMSE		-	28.46±3.55 (13 to 30)
HADS-A		-	6.27±3.92 (0 to 21)
HADS-D		-	4.85±2.05 (0 to 19)

SD: standard deviation; IC: confidence interval; freq.: frequency; MMSE: Mini Mental State Examination; HADS-A: Hospital Anxiety and Depression Scale, anxiety measure; HADS-D: Hospital Anxiety and Depression Scale, depression measure.

In the present paper, the discrete and regression-based norms are provided, either of which can be used by the examiner.

Using the discrete norms

The discrete norms, stratified by age and educational level, are shown in [Tables 2] and [3].

Table 2 Discrete norms: BICAMS scores stratified by age and educational level.

Age	BICAMS tests	Educational level
		1-11 years					>12 years
		Mean	±SD	Percentile			Mean	±SD	Percentile
				5th	10th	50th			5th	10th	50th
18-29 (n=134)	CVLT-II T1	8.1	2.8	4	4	8	7.7	2.2	5	5	7
	CVLT-II T2	10.4	2.6	5	6	11	10.6	2.3	7	8	10
	CVLT-II T3	12.2	2.3	8	8	13	12.4	2.5	8	9	13
	CVLT-II T4	12.8	2.5	7	9	13	13.4	2.5	10	11	14
	CVLT-II T5	13.4	2.2	8	10	14	14.1	2.2	10	11	15
	CVLT-II Total	56.8	10	36	40	57	58.2	9.3	41	45	59
	CVLT-II Total Rep.	3	4.7	-	-	-	3.3	3.7	-	-	-
	CVLT-II Total Intr.	1.5	2.8	-	-	-	1.5	4.1	-	-	-
	BVMT-R T1	7.4	2.9	2	3	8	7.3	2.9	2	3	7
	BVMT-R T2	9.3	2.4	4	6	10	9.5	2.2	4	7	10
	BVMT-R T3	10.3	2.3	4	7	12	10.6	1.8	6	8	11
	BVMT-R Total	27	7	10	17	27	27.5	5.9	15	19	28
	Oral SDMT	55	14	32	39	52	60	15.7	36	45	57
30-39 (n=181)	CVLT-II T1	7.6	2.4	4	5	8	7.4	2.2	4	5	7
	CVLT-II T2	10.2	2.4	6	7	10	10.6	2.6	6	7	10
	CVLT-II T3	11.7	2.4	8	8	12	12.4	2.8	6	9	13
	CVLT-II T4	12.3	2.4	8	9	12	13.1	2.4	9	10	13
	CVLT-II T5	12.9	2.1	10	10	13	13.9	2.2	9	11	15
	CVLT-II Total	54.8	9.1	35	39	53	57.4	10.2	38	44	58
	CVLT-II Total Rep.	5.6	5.4	-	-	-	4.8	5.9	-	-	-
	CVLT-II Total Intr.	1.1	1.8	-	-	-	1.3	2.1	-	-	-
	BVMT-R T1	6.1	3.6	2	2	5	6.7	2.9	2	3	6
	BVMT-R T2	8.4	3	3	4	8	9.3	2.4	5	6	10
	BVMT-R T3	10	2.5	5	6	11	10.5	1.9	6	8	11
	BVMT-R Total	24.4	8.1	10	13	26	26.6	6.3	15	19	27
	Oral SDMT	48.8	15	23	31	49	60.6	15.6	33	40	60
40-49 (n=102)	CVLT-II T1	6.5	2	4	4	6	7	2.2	4	5	7
	CVLT-II T2	9.8	2	7	7	10	9.9	2.6	5	6	10
	CVLT-II T3	11	2	7	8	11	11.7	2.4	8	9	11
	CVLT-II T4	11.9	2.5	5	9	12	12.8	2.3	8	10	13
	CVLT-II T5	12.9	1.8	10	10	13	13.2	2.3	9	10	14
	CVLT-II Total	52	8.3	35	30	51	54.5	9.3	39	42	55
	CVLT-II Total Rep.	5.2	3.7	-	-	-	5.4	6.4	-	-	-
	CVLT-II Total Intr.	1.1	1.3	-		-	1.2	1.9	-	-	-
	BVMT-R T1	5.6	3.3	0	1	5	5.7	2.5	2	2	6
	BVMT-R T2	8	2.9	2	4	9	8.4	2.7	3	5	9
	BVMT-R T3	9.1	31	1	4	11	10.1	2.4	4	6	11
	BVMT-R Total	22.8	8.4	6	12	23	24.2	6.6	10	13	26
	Oral SDMT	39.2	14.8	14	27	45	55.3	15.4	31	36	56

±SD: standard deviation; n: sample size; CVLT-II T1: California Verbal Learning Test second edition, trial 1; CVLT-II T2: California Verbal Learning Test second edition, trial 2; CVLT-II T3: California Verbal Learning Test second edition, trial 3; CVLT-II T4: California Verbal Learning Test second edition, trial 4; CVLT-II T5: California Verbal Learning Test second edition, trial 5; CVLT-II Total: California Verbal Learning Test second edition, Total sum score; CVLT-II Total Rep.: California Verbal Learning Test second edition, Total of Repetitions; CVLT-II Total Intr.: California Verbal Learning Test second edition, Total of Intrusions; BVMT-R T1: Brief Visuospatial Memory Test Revised, Trial 1; BVMT-R T2: Brief Visuospatial Memory Test Revised, Trial 2; BVMT-R T3: Brief Visuospatial Memory Test Revised, Trial 3; BVMT-R Total: Brief Visuospatial Memory Test Revised, Total Sum Score; Oral SDMT: Symbol Digit Modalities Test oral form.

Table 3 Discrete norms: BICAMS scores stratified by age and educational level (continuation).

Age	BICAMS tests	Educational Level
		1-11 years					>12 years
		Mean	±SD	Percentile			Mean	±SD	Percentile
				5th	10th	50th			5th	10th	50th
50-59 (n=105)	CVLT-II T1	7.1	2.2	4	5	7	6.9	2.4	3	4	7
	CVLT-II T2	9	2.4	5	6	9	9.3	2.4	6	6	9
	CVLT-II T3	10.6	2.4	7	8	10	11	2.6	6	7	12
	CVLT-II T4	11	2.7	5	7	12	12	2.3	6	9	12
	CVLT-II T5	12.3	2.7	8	9	13	13.4	2.3	9	10	14
	CVLT-II Total	50	9.6	35	32	49	52.5	8.3	38	42	53
	CVLT-II Total Rep.	4.9	4.4	-	-	-	4,1	3.7	-	-	-
	CVLT-II Total Intr.	1.4	2	-	-	-	1.3	2.3	-	-	-
	BVMT-R T1	3.5	2.3	0	1	4	5.4	3.0	0	1	5
	BVMT-R T2	5.4	3.2	0	1	5	7.3	2.8	3	4	7
	BVMT-R T3	7	3.3	2	2	8	9.3	2.4	3	5	10
	BVMT-R Total	16	8	4	6	18	22	7.3	8	13	24
	Oral SDMT	36.9	14	12	20	33	46.4	14.9	26	28	47
60-69 (n=53)	CVLT-II T1	6.2	2.1	1	4	6	6	1.7	3	3	6
	CVLT-II T2	8.5	2.7	2	6	9	8.7	1.6	6	6	9
	CVLT-II T3	9.8	2.8	2	7	10	10.4	2.4	5	6	11
	CVLT-II T4	10.3	3.4	2	7	11	12	2.2	8	8	12
	CVLT-II T5	10.9	3.3	1	6	12	13.2	2.2	8	10	13
	CVLT-II Total	45.6	12.9	30	31	48	50.2	7.9	32	37	52
	CVLT-II Total Rep.	6	3.2	-	-	-	6.3	6.2	-	-	-
	CVLT-II Total Intr.	1.5	2.2	-	-	-	1.6	1.7	-	-	-
	BVMT-R T1	3.8	2.7	0	1	3	4.2	3.4	1	2	4
	BVMT-R T2	5.5	2.8	1	2	6	6.5	3.5	1	3	6
	BVMT-R T3	6.5	3.7	0	1	7	8.3	2.8	1	5	8
	BVMT-R Total	15.8	8.4	1	5	16	22	9.1	2	8	17
	Oral SDMT	29.5	12.9	10	12	30	44	14.4	23	26	40
>70 (n=26)	CVLT-II T1	5.2	2.0	1	3	5	6.6	1.8	5	5	6
	CVLT-II T2	7.5	2.2	4	4	7	8.1	2.3	4	4	9
	CVLT-II T3	8.7	2.3	5	5	9	8.4	1.6	6	6	8
	CVLT-II T4	9.8	2.6	6	6	10	9.7	2.2	6	6	10
	CVLT-II T5	10.3	2.7	5	6	10	12.1	2.5	8	8	13
	CVLT-II Total	41.5	9.7	21	30	42	45	8,2	29	29	47
	CVLT-II Total Rep.	6.0	4.6	-	-	-	5	4.6	-	-	-
	CVLT-II Total Intr.	1.1	1.7	-	-	-	3.4	4.7	-	-	-
	BVMT-R T1	2.5	2.5	1	1	2	6	3.7	1	1	5
	BVMT-R T2	3.9	3.2	1	1	3	7.9	3.7	2	2	7
	BVMT-R T3	4.6	3.6	1	1	4	8.4	3.7	2	2	9
	BVMT-R Total	11.1	8.5	1	1	10	19	10.7	5	5	20
	Oral SDMT	23.7	9.8	9	9	26	40.1	8.6	23	26	37

±SD: standard deviation; n: sample size; CVLT-II T1: California Verbal Learning Test second edition, trial 1; CVLT-II T2: California Verbal Learning Test second edition, trial 2; CVLT-II T3: California Verbal Learning Test second edition, trial 3; CVLT-II T4: California Verbal Learning Test second edition, trial 4; CVLT-II T5: California Verbal Learning Test second edition, trial 5; CVLT-II Total: California Verbal Learning Test second edition, Total sum score; CVLT-II Total Rep.: California Verbal Learning Test second edition, Total of Repetitions; CVLT-II Total Intr.: California Verbal Learning Test second edition, Total of Intrusions; BVMT-R T1: Brief Visuospatial Memory Test Revised, Trial 1; BVMT-R T2: Brief Visuospatial Memory Test Revised, Trial 2; BVMT-R T3: Brief Visuospatial Memory Test Revised, Trial 3; BVMT-R Total: Brief Visuospatial Memory Test Revised, Total Sum Score; Oral SDMT: Symbol Digit Modalities Test oral form.

Using the regression-based norms

• Step 1: To obtain the predicted scores from raw scores, the examiner must convert the raw scores to scaled scores derived from the healthy controls using the data provided in [Table 4].

Table 4 Regression-based norms: BICAMS raw-to-scaled scores conversions.

Scaled scores	Raw Scores			Classification
	CVLT-II	BVMT-R	SDMT
1	<20			Extremely low
2	20-28
3	29-31	1-2	1-9
4	32-35	3-5	10-17
5	36-39	6-8	18-23	Borderline
6	40-41	9-12	24-29
7	42-44	13-17	30-36	Low average
8	45-48	18-20	37-43
9	49-52	21-23	44-49	Average
10	53-56	24-26	50-53
11	57-60	27-28	54-58
12	61-64	29-30	59-62	High average
13	65-66	31-32	63-68
14	67-69	33-34	69-74	Superior
15	70-71	35	75-79
16	72	36	80-93	Very superior
17	73-74		94-107
18	75		>107
19	>75

BICAMS: Brief International Cognitive Assessment for Multiple Sclerosis; SDMT: Symbol Digit Modalities Test; CVLT-II: California Verbal Learning Test second edition; BVMT-R: Brief Visuospatial Memory Test Revised.

• Step 2: To obtain the predicted scaled score (pss), the examiner must employ the predictive equation model. It is necessary to use the formula below (in gender, please use 1 for male and 2 for female) and the data from [Table 5] (final regression models and standard deviation of the residual for BICAMS measures).

Table 5 Final regression models and standard deviation of residuals for the BICAMS measures.

CVLT2 Residual SD		BVMT-R Residual SD		SDMT Residual SD
	2.527166		2.626665		2.48323
CVLT-2 Reg Model		BVMTR Reg Model		SDMT Reg Model
Predictor	B	Predictor	B	Predictor	B
(constant)	8.512324	(constant)	11.58455	(constant)	9.248778
Age	-0.14798	age	-0.14752	age	-0.01094
age[2]	0.001373	age[2]	0.000896	age2	-0.00086
Sex	0.176426	sex	-0.19042	sex	-0.4714
education	0.364315	education	0.22895	education	0.263055

BICAMS: Brief International Cognitive Assessment for Multiple Sclerosis; SD: standard derivation; CVLT-2: California Verbal Learning Test second edition; SDMT: Symbol Digit Modalities Test; BVMT-R: Brief Visuospatial Memory Test Revised; Reg. Model: Regression Model.

Predicted Scaled Score (pss) = Constant + β_age (age) + β_age (age²) + β_gender (gender) + β_ed. (ed.)

• Step 3: In sequence, the examiner must calculate the Z-score using the formula:

Z-score = (Actual scaled score - Predicted scaled score) / SD of residuals (provided in [Table 5]).

DISCUSSION

The present study aimed to provide both discrete and regression-based norms for the BICAMS. This study followed the recommendations of the international validation protocol for the BICAMS[10],[16] and is the first to publish in the Brazilian Portuguese the discrete and regression-based norms for BICAMS with age, gender and education corrections in the Brazilian population.

The BICAMS is a validated and reliable brief protocol for monitoring the CI of MS patients. In several countries, efforts are being made to establish it as a psychometrically valid and reliable tool that is internationally applicable. This protocol was optimized for small MS centers so it can be administered by health care professionals without specific expertise in neuropsychological testing and without high costs. Its clinical use allows large-scale international clinical trials to have a common outcome measure of cognitive functioning[23],[24]. It is recog nized that assessments and follow-ups of cognition should be as much as a priority as the evaluation of physical disability.

Discrete norms have their limitations and have come under criticism over in recent years, but they facilitate clinical use. Discrete norms are easier to use, but may distort demographic variables such as age, gender, and educational level, so they should be used with caution. Regression-based norms allow control of demographic influences on test performance.

In conclusion, our results provide the normative data of the BICAMS for use in the Brazilian context. Future studies are necessary to confirm its suitability for longitudinal assessments.

Conflict of Interest

There is no conflict of interest to declare.

Authors’ contributions:

CTS: was responsible for the study design, data collection, and writing; DAP, SEF, VDM, AAB, AS, PHRS, RHBB: contributed to the paper concept, literature review, and critical discussion.

Support

This work was partially supported by Coordination for the Improvement of Higher Education Personnel (CAPES).

• REFERENCES

• 1 Benedict RHB, Cookfair D, Gavett R, Gunther M, Munschauer F, Garg N. et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc 2006; 12 (04) 549-558 https://doi.org/10.1017/s1355617706060723
• 2 Chiaravalloti ND, DeLuca J. Cognitive impairment in multiple sclerosis. Lancet Neurol 2008; 7 (12) P1139-P1151 https://doi.org/10.1016/S1474-4422(08)70259-X
• 3 Guimarães J, Sá MJ. Cognitive dysfunction in multiple sclerosis. Front Neurol 2012; 3: 74 https://doi.org/10.3389/fneur.2012.00074
• 4 Rao SM, Leo GJ, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology 1991; 41 (05) 685-691 https://doi.org/10.1212/WNL.41.5.685
• 5 Kushwaha S, Suri P, Gupta M, Bala K. PO10-TU-64 Study of cognition and behavioural dysfunction in multiple sclerosis and correlation with MRI brain. J Neurol Sci 2009; 285 (Suppl. 01) S213 https://doi.org/10.1016/S0022-510X(09)70813-5
• 6 Strober LB, Christodoulou C, Benedict RHB, Westervelt HJ, Melville P, Scherl WF. et al. Unemployment in multiple sclerosis: the contribution of personality and disease. Mult Scler 2012; 18 (05) 647-653 https://doi.org/10.1177/1352458511426735
• 7 Langdon DW, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S. et al. Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS). Mult Scler 2012; 18 (06) 891-898 https://doi.org/10.1177/1352458511431076
• 8 Spedo CT, Frndak SE, Marques VD, Foss MP, Pereira DA, Carvalho LF. et al. Cross-cultural adaptation, reliability, and validity of the BICAMS in Brazil. Clin Neuropsychol 2015; 29 (06) 836-846 https://doi.org/10.1080/13854046.2015.1093173
• 9 Instituto Brasileiro de Geografia e Estatística Internet. Regional divisions of Brazil. IBGE; 2021 Nov 11 Available from: https://www.ibge.gov.br/en/geosciences/territorial-organization/regional-division/21536-regional-divisions-of-brazil.html?=&t=o-que-e
• 10 Benedict RHB, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S. et al. Brief International Cognitive Assessment for MS (BICAMS): international standards for validation. BMC Neurol 2012; 12: 55 https://doi.org/10.1186/1471-2377-12-55
• 11 Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale: An updated literature review. J Psychosom Res 2002; 52 (02) 69-77 https://doi.org/10.1016/S0022-3999(01)00296-3
• 12 Folstein MF, Folstein SE, Mchugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12 (03) 189-198 https://doi.org/10.1016/0022-3956(75)90026-6
• 13 Smith A. The symbol-digit modalities test: a neuropsychologic test for economic screening of learning and other cerebral disorders. Learn Disord 1968; 3: 83-91
• 14 Lezak MD, Howieson DB, Loring DW, Hannay HJ, Fischer JS. Neuropsychological assessment. 4. Oxford University Press; 2004
• 15 Strober L, Englert J, Munschauer F, Weinstock-Guttman B, Rao S, Benedict RHB. Sensitivity of conventional memory tests in multiple sclerosis: comparing the Rao Brief Repeatable Neuropsychological Battery and the Minimal Assessment of Cognitive Function in MS. Mult Scler 2009; 15 (09) 1077-1084 https://doi.org/10.1177/1352458509106615
• 16 Benedict RHB, Fischer JS, Archibald CJ, Arnett PA, Beatty WW, Bobholz J. et al. Minimal neuropsychological assessment of MS patients: a consensus approach. Clin Neuropsychol 2002; 16 (03) 381-397 https://doi.org/10.1076/clin.16.3.381.13859
• 17 Delis DC, Kramer JH, Kaplan E, Ober BA. Manual for the California Verbal Learning Test, (CVLT-II). 2. San Antonio (TX): The Psychological Corporation; 2000
• 18 Benedict RHB. Effects of using same- versus alternate-form memory tests during short-interval repeated assessments in multiple sclerosis. J Int Neuropsychol Soc 2005; 11 (06) 727-736 https://doi.org/10.1017/S1355617705050782
• 19 Chiaravalloti ND, Moore NB, Nikelshpur OM, DeLuca J. An RCT to treat learning impairment in multiple sclerosis: the MEMREHAB trial. Neurology 2013; 81 (24) 2066-2072 https://doi.org/10.1212/01.wnl.0000437295.97946.a8
• 20 Benedict RHB, Groninger L, Schretlen D, Dobraski M, Shpritz B. Revision of the brief visuospatial memory test: Studies of normal performance, reliability, and, validity. Psychol Assess 1996; 8 (02) 145-153 https://doi.org/10.1037/1040-3590.8.2.145
• 21 Goretti B, Niccolai C, Hakiki B, Sturchio A, Falautano M, Minacapelli E. et al. The brief international cognitive assessment for multiple sclerosis (BICAMS): normative values with gender, age and education corrections in the Italian population. BMC Neurol 2014; 14: 171 https://doi.org/10.1186/s12883-014-0171-6
• 22 Smerbeck AM, Parrish J, Yen EA, Weinstock-Guttman B, Hoogs M, Serafin D. et al. Regression-based norms improve the sensitivity of the National MS Society Consensus Neuropsychological Battery for Pediatric Multiple Sclerosis (NBPMS). Clin Neuropsychol 2012; 26 (06) 985-1002 https://doi.org/10.1080/13854046.2012.704074
• 23 Parmenter BA, Testa SM, Schretlen DJ, Weinstock-Guttman B, Benedict RHB. The utility of regression-based norms in interpreting the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc 2010; 16 (01) 6-16 https://doi.org/10.1017/S1355617709990750
• 24 Sandi D, Rudisch T, Füvesi J, Fricska-Nagy Z, Huszka H, Biernacki T. et al. The Hungarian validation of the brief international cognitive assessment for multiple sclerosis (BICAMS) battery and the correlation of cognitive impairment with fatigue and quality of life. Mult Scler Relat Disord 2015; 4 (06) P499-P504 https://doi.org/10.1016/j.msard.2015.07.006

Publication History

Received: 26 November 2020

Accepted: 26 January 2021

Article published online:
30 January 2023