Journal of Psychiatry and Cognitive Behavior

Volume 2017; Issue 01
7 Sep 2017

Test-Retest Reliability of a Computerized Neuropsychological Test Battery: A Cross-Sectional Study Assessing Cognition in Healthy Young and Old Adults, and Stroke Survivors

Research Article

Jinal Vora1, Rini Varghese1, Victor Patron2,3, Sara Weisenbach4,5, Tanvi Bhatt1*

1Department of Physical Therapy, University of Illinois at Chicago, Chicago, USA
2Department of Psychiatry, University of Illinois at Chicago, Chicago, USA
3Department of Psychiatry, Rush University, Chicago, USA
4Department of Psychiatry, University of Utah, Salt Lake City, USA
5Research & Development Program, VA Salt Lake City, Salt Lake City, USA

*Corresponding author:Tanvi Bhatt, Department of Physical Therapy, University of Illinois at Chicago, Chicago, USA, Tel: +1 3123554443; Fax: + 13129964583; Email:

Received Date: 01 March, 2017; Accepted Date:13 March, 2017; Published Date: 20March, 2017







Suggested Citation





Dual-task methodology is widely utilized to measure sharing of attentional resources between motor and cognitive functions. Computerized neuropsychological testing is an advanced approach of cognitive assessment. The traditional paper-and-pencil tests, although are valid and reliable, have certain drawbacks hence limiting their overall usability, particularly in clinical settings. This study aimed to establish the test- retest reliability of a computerized, custom-designed cognitive test battery in healthy young and older adults and chronic stroke survivors.




Healthy young (n=15) and older (n=15) adults and adult chronic stroke survivors (n=15) were tested for the following domains via DirectRT, Empirisoft


  • visuo-motor function (Spot & Click, SC)
  • associative memory, via a paired associative learning task (Number & Position, NP)
  • semantic fluency, (Category Naming, Cat.N)
  • information processing speed/response inhibition (Color Naming, CN)
  • Discriminant Decision Making (Unveil the Star, US)
  • visual working memory using 1-back and 2-back tasks (Triangle Tracking, TT). The outcome variables consisted of reaction time, accuracy, total time of completion and total number of responses.




A good to excellent reliability for all the six tasks (p<0.05) for each of the three groups was observed. Results demonstrate, that stroke survivors performed worse than healthy young and older adults whereas both the healthy young and older adults showed similar performance across different tests.




Results indicated that these computerized cognitive measures were highly reproducible and reliable. Such testing could be easily implemented by clinicians for assessing cognition and incorporated in dual-task testing and training paradigms.


Keywords: Cognition; Dual Task;Neuropsychological Testing; Test-Retest Reliability; Stroke



Neurocognitive decline is a growing healthcare concern that can result in loss of functional independence in healthy aging baby boomers and individuals with neurological diagnoses [1].  More than 5.1 million individuals in the United States are estimated to be living with cognitive impairment [2]. The medical costs for older adults with Mild Cognitive Impairment (MCI) are 44% higher than those for non-impaired older adults [3]. Due to structural changes in the prefrontal cortex parietal and medial temporal regions including hippocampus, cognitive functions most commonly affected in these individuals include executive function, information processing, and working and associative memory [4]. In addition to age-related cognitive changes, individuals with long-standing neurologic disorders, such as stroke, may retain permanent damage to brain areas affecting cognitive, sensory, and motor functions [5].


Cross-sectional studies show that 64% of stroke survivors have cognitive impairments unrelated to dementia [6]. Furthermore, among individuals presenting with MCI post-stroke, the chances of reversibility of cognitive impairment decline with the chronicity of the condition [7,8]. Recent findings suggest that higher cortical centers may contribute to poor motor control during performance of daily functional motor activities like walking, grocery shopping, and driving by sharing the same attentional resources with cognitive functions such as working memory, spatial and discriminant decision making and information-processing [9,10]. This has been observed through dual-task paradigms focusing on simultaneous performance of a motor and a cognitive task. Studies have shown that dual-tasking results in deterioration of performance in either motor and/or cognitive tasks, which is described as Cognitive Motor Inference (CMI) [11].There are various potential patterns of interference described by Plummer et al.(2013) namely


  • No interference, where the performance on either task does not change relative to single-task performance.
  • Cognitive-related motor performance, where cognitive performance remains stable whereas motor performance deteriorates.
  • Motor-related cognitive interference, where motor performance remains stable and the cognitive performance deteriorates.
  • Mutual interference, where performance on both tasks deteriorates [12,13].


CMI result from competing demands of both motor and cognitive tasks for accessing the limited and probably shared processing resources within the brain [14].


In older adults, it has been observed that motor performance is significantly altered with concurrent performance of a secondary cognitive task (dual-task) such as increased postural sway and decreased gait speed and step lengths [15,16]. Such findings of deterioration in gait and balance performance are evident in stroke survivors, as well. Dual task assessments could thus provide a platform to understand the interaction between cognitive and motor systems in healthy and neurologically-affected individuals [12].


Although there is evidence suggesting a critical role of cognition in facilitating motor performance, cognitive testing and interventions have not been routinely utilized in clinical rehabilitation settings. Studies indicate that neural plasticity endures across the lifespan, and continuous cognitive stimulation is important for enhancement and maintenance of cognitive functioning [17]. However, prior to providing cognitive or cognitive-motor interventions, an accurate and comprehensive neuropsychological assessment is fundamental to identify specific cognitive domains most severely impacted by aging and/or neurologic processes.


Conventional paper-and-pencil measures used to assess cognitive function tend to be lengthy, require specialized training, and are prone to manual error. They are also not feasible for use in cognitive-motor dual task paradigms. Computerized Cognition Testing (CCT) is gaining popularity and offers an alternative to some of the traditional paper-and-pencil testing measures. CCT has potential advantages over conventional testing methods as it renders precise stimulus control, consistency in administration and scoring, visually appealing interfaces, cost efficiency, ability to develop large and accurate databases and ease of administration of cognitive-motor testing paradigms [18]. Computerized cognitive testing also helps to assess and monitor cognition in large sample size [19].  However most of the commercially available neuropsychological test batteries are generally lengthy and require several sessions of testing as they assess multiple domains of cognition thoroughly [20-23]. Without understanding the burden the patients undergo which may eventually affect the test outcome measures.  Keeping this in mind and after extensive research we developed a customized computerized cognitive test battery consisting of six task assessing the six-main domain of cognition namely, visuo-motor function, verbal fluency, executive function, discriminant decision making memory, working memory and associative memory and the average total time to complete this battery was 40 minutes. We choose to evaluate the test battery administered via the DirectRT Empirisoft over other commercially available tests as it is affordable, user friendly, allows to develop customized test stimuli and easy to modulate tests with increasing difficult levels and allow subsequent interpretation of a variety of cognitive tests without the need for specialized training.


Nonetheless, there can be certain challenges associated with use of computers to assess cognitive function in older or post-stroke population. Hence a good interface design and simple cognitive tests were created that could improve the usability of computerized testing in the above-mentioned population and in turn increase the reliability and validity of the cognitive data [24,25].In a previous study, we examined the reliability and concurrent validity of a computerized cognitive test battery administered using DirectRTEmpirisoft among healthy young adults [26]. While the previous study reported positive findings, we concurred that the battery’s reliability needs to be examined in other populations prone to cognitive decline.


The primary aim of this study was to establish the test-retest reliability of a custom-designed, Computerized Cognitive Test battery (CCT) in post-stroke survivors, healthy older adults and young adults, using the commercially available Direct RT Empirisoft, to determine its potential for use in clinical or research settings. The secondary aim was to understand the effect of aging and neurologic disorder due to stroke on cognitive functioning amongst healthy older individuals and stroke survivors. For this aim, we hypothesized that cognition would be significantly impaired in the participants with stroke as compared to healthy older adults and young adults.






Fifteen adults with chronic hemiparetic motor stroke as confirmed by the physician (age: 58.9± 6.9, years of education: 14.2± 2.7), fifteen healthy older adults (age: 64.4±5.1, years of education: 16.2± 2.3), and fifteen healthy young adults (age: 23.86 ± 1.35, years of education: 16.8 ± 1.47) participated in this study. Participants were recruited via informational flyers posted across the University of Illinois at Chicago Medical campus. Exclusion criteria for participants with chronic stroke was severe receptive or expressive dysphasia, which would limit their participation in the assessment as determined via the scores obtained on Mississippi Aphasia Screening Test [27]. Individuals under medications impacting cognition or having comorbid psychiatric and medical conditions were also excluded.  Healthy older and young adults were included if they had no self-reported neurological disorder or injury that could affect sensory, motor or cognitive functioning. The study was approved by the Institutional Review Board of the University of Illinois at Chicago, and informed consent was obtained from all participants.


Prior to testing, individuals premorbid IQ score was assessed using the WRAT-4 reading test (total score out of 70) after which participants completed the computerized neurocognitive test battery administered using DirectRT, Empirisoft[28].


All testing was conducted in a silent room in order to avoid any external disturbances. For computerized testing, a screen was placed in front of the participant and a headset with microphone was used to record the responses. Each test was preceded by an instruction slide and instructions were also repeated by the examiner, after which the participant was asked to press a key when he or she was ready to start the test. Each participant underwent two testing sessions separated by 10-12 days’ interval. During each session, all the tests from the computerized cognitive test battery were administered in a randomized order i.e. with respect to the order of the tests administered.


Neuropsychological Test Battery


Six domains of cognition (visuo-motor function, verbal fluency, executive function, discriminant decision making memory, working memory and associative memory) were included in the testing protocol. Visuo-motor function was assessed by measuring the response latencies after the stimulus is presented to the examinee i.e. the Spot & Click task. Both, Simple Reaction Time (SRT) and Choice Reaction Time (CRT) was assessed. The participant was presented with a stimulus (yellow circle) and asked to respond by pressing the corresponding key on the number pad depending on the position of the stimulus. Measurement of executive function consisted of the classic strop paradigm, which measures processing speed and response inhibition. The computerized version of the test (Color Naming) consisted of two conditions. In the first condition the participant was asked to read the words that were printed in the same color ink (congruent condition). Condition 2 required the participant to name the ink (color) in which the color word was printed. The actual word frequently differed from the ink in which it was printed (incongruent condition). The total time taken to complete the task was recorded. Unveil the Star task assessed the manipulation and retention of visuo-spatial information and discriminant decision making. In this task, the participant was required to search for a blue star in multiple red boxes without clicking on the same box twice where the star was previously found. There were three increasing levels-of-difficulty as the task progressed. The total time taken to complete the task, along with the total number of errors was recorded. Test of verbal fluency included the Category Naming task in which participants’ semantic fluency was assessed.  Participants were provided a category cue (i.e. animals, boys’ names, or countries) and were given one minute to provide as many words as possible relating to that category. They were instructed to avoid repeating the same responses. The voice responses were recorded by the computer via the microphones. Working memory was also assessed using the Triangle Tracking task in which the participant was presented with a sequence of stimuli and asked to indicate when the current stimulus matches the one from n-steps earlier in the sequence. We used 1 and 2 steps earlier for the current protocol. Lastly, the Number & Position task was administered to assess associative memory, where the participant was presented with a slide displaying numbers in a grid and the participant was asked to memorize the position of the number (i.e. 7- center, 3- Upper left, 1- lower right) after which a slide with the single number in the center was presented and the participant was asked to recall the position of the number from the previously presented slide. The outcome variables for each of the computerized neurocognitive test battery are presented in Table 1.


Statistical Analyses


Descriptive statistics (mean ± SD) were performed for all cognitive test variables. Intra-Class Correlation Coefficients (ICC) was used to determine the reliability of each of the computerized tests in the cognitive battery. Test-retest reliability was characterized as excellent (ICC >0.8), good (ICC 0.6-0.79), moderate (ICC 0.4-0.59), fair (ICC 0.2-0.39) and poor (ICC<0.2)[29]. The level of agreement between the two testing sessions is represented using Bland Altman plots. One-sample t-test was used to analyze whether the bias (difference) between the mean score of the two sessions was significantly different from zero. ANCOVA was performed to analyze the significant differences between the groups in cognitive test variables using the premorbid IQ score and Years of education as covariates among the three groups followed by post-hoc Tukey’s test. The statistical significance was set at p<0.05. All statistical analyses were performed using Statistical Package for Social Sciences (SPSS), version 22.0 (Chicago, IL version 22).




Demographic data


Demographic comparisons between the three groups (i.e., stroke, healthy older adults, and healthy young adults) are shown in Table2together with the neurological data.




Paired t-tests showed no significant difference between the two testing sessions among the different variables. Test-retest reliability data revealed significant Intra-Class Correlation (ICC) with 95% CI for each of the Plots which displays the level of agreement between the testing sessions in Figure 1.


Group Effects


The results of the ANCOVA on the Computerized Cognitive Test (CCT) battery showed that the difference in the cognitive test between the stroke group, the healthy older adults group, and the healthy young adults group were statistically significant (p<0.05), as presented in Table 4.


Premorbid IQ scores and years of education were significant covariates for the Reaction Time test variables.Tukey’s HSD test revealed significant differences in the CCTs between the three groups. The stroke group showed a significantly longer reaction time on Spot & Click task, longer time for completion for Color Naming task than the healthy older group (p< 0.05) and healthy young group (p< 0.05). The stroke group presented significant more errors for Unveil the Star task as compared to healthy young group (p<0.05). The stroke group presented significantly lower number of correct responses and decreased accuracy for the Triangle Tracking task and Number & Position task as compared to the healthy older adult group (p<0.05) and healthy young group (p<0.05). The healthy older adult group showed a significantly longer reaction time for the Spot and Click task and an increased time for completion for the Unveil the Star task, as compared to the healthy young adult group (p<0.05) as presented in Table 5 and Figure 2.




Overall, the findings from our study indicate that the Computerized Cognitive Tests (CCTs) administered via DirectRTEmpirisoft are moderately reliable measure of cognitive functioning across two testing sessions in chronic stroke survivors, healthy older adults and young adults. Further, the secondary finding of our study was that the CCTs were able to detect difference in cognition between a small sample of stroke survivors, healthy older adults and young adults.


Test-retest reliability


The test-retest reliability correlations measured with Intra-Class Correlation Coefficient (ICC) for all of the computerized cognitive tasks assessed across two testing sessions ranged from 0.75-0.98 for all of the groups together. On further analyzing the reliability coefficient for individual groups, a high test-retest reliability coefficient was obtained for the young and older adults as compared to chronic stroke survivors for both simple and choice reaction task. The range of the reliability coefficients in this study across all the groups for reaction time was consistent with the previously reported ranges i.e. ICC=0.60-0.98 [30-32].

Furthermore, the high test-retest coefficients reported in the study were not biased depending on the practice effects because the stimuli across both testing sessions were unpredictable for the participants being tested as the stimuli presented were random. It has been proposed by Lowe and Rabbit [30] that reaction time is a non-strategic task and therefore might not result in forming a memory as to when the stimuli will appear. This task can thus yield high test-retest correlation, suggesting that the likelihood of practice effects was reduced as a result of unexpected stimuli.


For the Category Naming task (verbal fluency), a high test-retest reliability coefficient for all groups together i.e. ICC=0.89 and for individual groups was attained. The Unveil the Star task (Discriminant Decision Making) and Color Naming task (processing speed/response inhibition) demonstrated high ICCs for the groups together. Moderate test-retest correlation for total time taken to complete Unveil the Star task was observed for individual groups. Examination of the data suggests that participants required less time to complete the task during the second session as compared to the first session possibly due to familiarity with the test. Similar results were obtained for the Triangle Tracking task (i.e., working memory) and the Number & Position task (i.e., associative memory) wherein moderate test-retest reliability was observed for all three groups together. Hence, we suggest that working memory tasks are usually strategy-driven, and accurate results in order to interpret test-retest reliability scores can be provided only when they are novel as performance on them can improve once the participant discovers an optimal strategy for it [30] or alternatively the stimuli presented in the some of the memory tasks were learned and remembered between the sessions resulting into moderate test-retest reliability for all three groups together[33].


Group Differences in cognitive abilities


Our secondary findings focused on understanding mean group differences and interpreting which cognitive domain is primarily impaired post-stroke. The reaction time recorded for the choice reaction time task and the time difference calculated between simple and choice reaction time tasks (CRT-SRT) demonstrated higher means for stroke survivors compared to healthy older and young adults. A significant difference in the reaction time was observed between the older adults and young adults. Reaction time has been extensively studied in the literature and is found to increase with age [34,35]. It has also been suggested by Brein et al that reaction time is one of the most sensitive markers of structural and functional deterioration in the aging central nervous system which is further hampered if there is underlying neurological insult such as stroke [36].


The stroke group demonstrated greater number of errors for Unveil the Star task (discriminant decision making) as compared to the young adults. They also demonstrated decreased accuracy on the Triangle Tracking task (working memory) and the Number & Position task (associative memory) as compared to older adults and young adults. For Unveil the Star task, the older adults and stroke groups required more time to complete the task as compared to young adults. One explanation for this result can be related to the increased complexity inherent in the task, as the individual is required to follow strategic sequence in which he/she successfully avoids the boxes on which they have already clicked. Further we may suggest that stroke affecting prefrontal and frontal areas could have impacted processing within the working memory system, thereby requiring more time and have less accuracy in completing the task but more information with respect to the site of the lesion and affected lobe is warranted.


Studies have suggested that executive function tasks could be as equally complex as the working memory tasks, demanding greater attentional resources for information processing and response inhibition by suppressing task-irrelevant information [9,37] resulting in increased time of completion for the incongruent slide of the Color Naming task as compared to the congruent slide. For the Category Naming task, in which the participants were asked to recall words belonging to a particular semantic category (e.g. animal), significant difference for number of correct responses was observed between chronic stroke survivors and healthy older and young adults. The stroke group demonstrated decreased number of correct responses for the Category Naming task (semantic fluency) as compared to the other two groups. This suggest that that verbal fluency is a frontally mediated cognitive function and hemiparetic stroke can result in poor performance on this task [38].


Notwithstanding these results, the study had certain limitations. The overall sample size was small, and reliability measurements should be repeated in a larger sample and also try to limit practice effects due to familiarity to the paradigm.  Furthermore, for future implication we would like to incorporate not only the type but also the site of lesion for a better understanding of which cognitive function is affected more by comparing the sites of the cerebral lesion. Also, the CCTs can be further explored in people suffering from MCI, dementia or any other cognitive impairments.


To conclude, the results from the study suggest that the customized computerized cognitive test battery is a reliable tool to assess cognitive function.  The custom developed CCT’S may be a good tool to capture more subtle changes in cognitive functioning, relative to paper-and-pencil clinical global cognition tests such as the Montreal Cognitive Assessment which can be useful as screening tool [39]. Computerized measures also require less training for administration and can be employed in busy clinical settings without undue burden on staff. Computerized batteries can be used as either screening instruments to identify patients requiring further evaluation by a neuropsychologist, or as part of a standard neuropsychological assessment. Lastly, CCTT’s can be employed as dual-task paradigms for cognitive-motor rehabilitation in clinical settings. Future studies should establish sensitivity and specificity of this CCT for identifying Mild Cognitive Impairment and dementia among patients with stroke and other neurological problems.




This work was supported by a pilot grant the Midwest Royal Centre for Health Promotion and Translation under Grant P30AG022849 awarded to Dr. Tanvi Bhatt.


Declaration of interest


The authors report no conflicts of interest.The project described was supported by Grant Number P30AG022849 from the National Institute on Aging. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health.



  1. Deary IJ, Corley JGow AJHarris SEHoulihan LM, et al. (2009) Age-associated cognitive decline. Br Med Bull 92: 135-152.
  2. Hebert L.E, Scherr PABienias JLBennett DAEvans DA (2003) Alzheimer disease in the US population: prevalence estimates using the 2000 census. Arch Neurol  60: 1119-1222.
  3. Zhu C.W, Sano MFerris SHWhitehouse PJPatterson MB (2013) Health-related resource use and costs in elderly adults with and without mild cognitive impairment. J Am Geriatr Soc 61: 396-402.
  4. Simons JS, HJ Spiers (2003) Prefrontal and medial temporal lobe interactions in long-term memory. Nat Rev Neurosci 4: 637-648.
  5. Ku J, Lee JHHan KKim SIKang YJ, et al. (2009) Validity and reliability of cognitive assessment using virtual environment technology in patients with stroke. Am J Phys Med Rehabil 88: 702-710.
  6. Jin Y.P, Di Legge SOstbye TFeightner JWHachinski V (2006) The reciprocal risks of stroke and cognitive impairment in an elderly population. Alzheimers Dement 2: 171-178.
  7. Rasquin S.M, J. Lodder, and F.R. Verhey (2005) Predictors of reversible mild cognitive impairment after stroke: a 2-year follow-up study. J Neurol Sci 229-230: 21-25.
  8. Schaapsmeerders P, Maaijwee NAvan Dijk EJRutten-Jacobs LCArntz RM, et al.(2013) Long-term cognitive impairment after first-ever ischemic stroke in young adults. Stroke 44: 1621-1628.
  9. Patel P, T Bhatt (2014) Task matters: influence of different cognitive tasks on cognitive-motor interference during dual-task walking in chronic stroke survivors. Top Stroke Rehabil 21: 347-357.
  10. Owsley C, Ball KSloane MERoenker DLBruni JR(1991) Visual/cognitive correlates of vehicle accidents in older drivers. Psychol Aging 6: 403-415.
  11. Abernethy B (1988) Dual-task methodology and motor skills research: some applications and methodological constraints. Journal of Human Movement Studies 14: 101-132.
  12. Plummer P, Eskes GWallace SGiuffrida CFraas M, et al. (2013) Cognitive-motor interference during functional mobility after stroke: state of the science and implications for future research. Arch Phys Med Rehabil 94: 2565-2574.
  13. Lundin-Olsson L, L. Nyberg and Y. Gustafson (1998) Attention, frailty, and falls: the effect of a manual task on basic mobility. J Am Geriatr Soc 46: 758-761.
  14. Al-Yahya E, Dawes H, Smith L, Dennis A, Howells K, et al. (2011) Cognitive motor interference while walking: a systematic review and meta-analysis. Neurosci Biobehav Rev 35: 715-728.
  15. Snijders A.H, C. Verstappen, M. Munneke, B. R. Bloem(2007) Assessing the interplay between cognition and gait in the clinical setting. J Neural Transm (Vienna) 114: 1315-1321.
  16. Brauer SG, M. Woollacott, A. Shumway-Cook(2002) The influence of a concurrent cognitive task on the compensatory stepping response to a perturbation in balance-impaired and healthy elders. Gait Posture 15: 83-93.
  17. Park D.C, Polk TAMikels JATaylor SFMarshuetz C(2001) Cerebral aging: integration of brain and behavioral models of cognitive function. Dialogues Clin Neurosci 3: 151-165.
  18. Gualtieri C.T and L.G. Johnson (2006) Reliability and validity of a computerized neurocognitive test battery. CNS Vital Signs. Arch Clin Neuropsychol 21: 623-643.
  19. Tornatore J.B, Emory Hill, Jo Anne Laboff, Mary E. McGann(2005) Self-administered screening for mild cognitive impairment: initial validation of a computerized test battery. J Neuropsychiatry Clin Neurosci 17: 98-105.
  20. Hommel M, Miguel STNaegele BGonnet NJaillard A(2009) Cognitive determinants of social functioning after a first ever mild to moderate stroke at vocational age. Journal of Neurology, Neurosurgery & Psychiatry 80: 876-880.
  21. Sachdev PS (2003) The neuropsychological profile of vascular cognitive impairment in stroke patients. in The Eleventh International Congress.
  22. Särkämö T, Tervaniemi MLaitinen SForsblom ASoinila S, et al.(2008) Music listening enhances cognitive recovery and mood after middle cerebral artery stroke. Brain 131: 866-876.
  23. Jaillard A, Naegele BTrabucco-Miguel SLeBas JFHommel M (2009) Hidden dysfunctioning in subacute stroke. Stroke 40: 2473-2479.
  24. Van Gerven, P.W, Fred Paas, and H.K. Tabbers (2006) Cognitive aging and computer-based instructional design: Where do we go from here? Educational Psychology Review 18: 141-157.
  25. Acevedo A, Loewenstein DA (2007) Nonpharmacological cognitive interventions in aging and dementia. J Geriatr Psychiatry Neurol 20: 239-249.
  26. Vora J (2016) Test-retest reliability and validity of a custom-designed computerized neuropsychological cognitive test battery in young healthy adults. Journal of Psychology and Cognition 1: 11-19.
  27. Nakase-Thompson R(2004) The Mississippi Aphasia Screening Test. The Center for Outcome Measurement in Brain Injury 71-85.
  28. Jarvis BG, (200x) DirectRT (Version 200X.X.X) [ Computer Software], 2014: New York, NY: Empirisoft Corporation.
  29. Guo C.C, Florian Kurth, Juan Zhou, Emeran A. Mayer, Simon B Eickhoff, et al.(2012) One-year test-retest reliability of intrinsic connectivity network fMRI in older adults. Neuroimage 61: 1471-1483.
  30. Lowe C, Rabbitt P (1998) Test/re-test reliability of the CANTAB and ISPOCD neuropsychological batteries: theoretical and practical issues. Cambridge Neuropsychological Test Automated Battery. International Study of Post-Operative Cognitive Dysfunction. Neuropsychologia 36: 915-923.
  31. Register-Mihalik J.K, Kontos DL, Guskiewicz KM, Mihalik JP, Conder R, et al. (2012) Age-related differences and reliability on computerized and paper-and-pencil neurocognitive assessment batteries. J Athl Train 47: 297-305.
  32. Collie A, Maruff PMakdissi MMcCrory PMcStephen M, et al. (2003) CogSport: reliability and correlation with conventional cognitive tests used in postconcussion medical evaluations. Clin J Sport Med 13: 28-32.
  33. Fredrickson J, Maruff PWoodward MMoore LFredrickson A, et al.(2010) Evaluation of the usability of a brief computerized cognitive screening test in older people for epidemiological studies. Neuroepidemiology 34: 65-75.
  34. Clarkson PM (1978) The effect of age and activity level on simple and choice fractionated response time. Eur J Appl Physiol Occup Physiol 40: 17-25.
  35. Gottsdanker R (1982) Age and simple reaction time. J Gerontol 37: 342-348.
  36. Birren JE, Woods AM, Williams MV (1980) Behavioral slowing with age: Causes, organization, and consequences 293-308.
  37. Long DL, Prat CS (2002) Working memory and stroop interference: an individual differences investigation. Mem Cognit 30: 294-301.
  38. Brady CB, Spiro A,McGlinchey-Berroth RMilberg W, et al.(2001) Stroke risk predicts verbal fluency decline in healthy older men: evidence from the normative aging study. J Gerontol B Psychol Sci Soc Sci 56: 340-346.
  39. McDonnell MN, Bryan JSmith AEEsterman AJ(2011) Assessing cognitive impairment following stroke. J Clin Exp Neuropsychol 33: 945-953.


Figure 1: Bland-Altman plots displaying limits of agreement: mean (solid line) ±1.96 SD (dotted line) of the difference between test session 1 and test session 2 values for the variables: (a) number of correct responses; (b & f) total time completion; (c) total errors (d) choice reaction time (CRT); and (e) accuracy (%).(The represents the stroke group, represents healthy-older adults &represents the healthy young group).



Figure 2: Plots displaying means of each outcome variables for test session 1 and 2 which are not significantly different from each other (p >0.05). a) Choice reaction time b) Total time completion for Color Naming task- incongruent slide c) Unveil the Star task respectively. d) Accuracy (%) for Number & Position task e) Accuracy (%) for Triangle Tracking (2- back) f) number of correct responses for the category naming task.



Test Name Cognitive function assessed Outcome variable
Spot & Click Visuo-motor Reaction Time
Category Naming Semantic verbal Fluency Number of correct responses
Color Naming Response inhibition/processing Speed Total time of completion
Unveil the Star Discriminant Decision Making Total time of completion
Total number of errors.
Triangle Tracking Working Memory Number of correct responses
Number & Position Associative Memory Accuracy (%)


Table 1:Computerized cognition tests assess for test-retest reliability and outcome variables.


Stroke patients mean(SD)/ % Healthy Older adults mean(SD) Healthy Young mean(SD)
N= 15 N=15 N=15
Age (years) 58.9(6.9) 64.4(5.1)$ 23.86(1.3)
Gender (M/F) 07-Aug 08-Jul 06-Sep
Premorbid IQ (%) 55.8/70 (79.8%) 64.8/70 (92.6%)* 64.6/70 (92.3%)#
Years of Education 14.2(2.7) 16.2(1.7)* 16.8(1.5)#
Time since stroke (years) 10.29(5.99)
Stroke type 58.82/41.18
(% I/H)
Note: I- ischemic, H- Hemorrhagic
$ Significant difference in mean age between the stroke group and the healthy older adult group.
Significant difference in means for Premorbid IQ and Years of Education (p ≤ 0.05).
* Difference in means between stroke and Healthy older adults statistically significant (p < 0.05).
# Difference in means between stroke and Healthy young statistically significant (p < 0.05).


Table 2: Demographics and disease characteristics of chronic stroke patients, older adults and young adults.


Computerized test Test Variables Stroke patients mean(SD) Healthy Older adults mean(SD) Healthy Young mean(SD) F value P
    (N=15) (N=15) (N= 15)    
Spot & Click CRT (ms)* 1347.1 ± 333.7 994.4 ±192.1 694.2 ± 150 17.6 <0.001
CRT-SRT (ms)* 858. ± 299.1 565.8 ± 146.7 356.9 ± 116.9 14.9 <0.001
Category Naming # of responses 16.8 ± 5.8 23.6 ± 6.1 22.20 ± 5.8 5.5 <0.001
Color Naming Incongruent (total time, s) 63.9 ± 24.1 44.5 ± 6.5 33.6 ± 7.8 15.4 <0.001
Unveil the Star Total time of completion(s) 98.3 ± 34.1 73.9 ± 26.1 48.32 ± 12.56 14.1 <0.001
Error (%) -1451.6 ± 28.2 -1464.5 ± 22.2 -1475.8 ± 25.7 3.3 0.04
Triangle Tracking 2-back Accuracy (%) 45 ± 22.5 73.3 ± 17.5 78.33 ± 17.3 12.9 <0.001
Number & Position # of correct responses 84 ± 19.2 96.66 ± 7.4 98.33 ± 4.4 6.1 <0.001
Note: *Premorbid IQ Score and years of education were significant covariates (p<0.05).


Table 4: Computer generated test scores for stroke patients, healthy older adults, and healthy young adults.


Computerized test Test Variables Stroke vs. Older adults mean diff Stroke vs. Healthy young mean diff Healthy young vs. Older adults mean diff Stroke vs. Older adults Stroke vs. Healthy young Healthy young vs. Older adults
(95%CI) (95%CI) (95%CI)
Spot & Click CRT (ms)* 352.72 652.85 300.13 p=0.01 p<0.001 p=0.04
CRT-SRT (ms)* 292.22 50.1.10 208.88 p<0.001 p<0.001 p=0.02
(111.3-473.1) (320.2-681.9) (28.0-386.7)
Category Naming # of responses -6.8 -5.4 -1.4 p<0.001 p=0.04 p=0.79
(-12.1– -1.5) (-10.2 – -1.2) (-6.7 – 3.9)
Color Naming Incongruent (total time, s) 19.44 30.36 -10.91 p <0.001 p <0.001 p = 0.13
Unveil the Star Total time of completion(s) 24.42 50.05 -25.62 p=0.03 p<0.001 p=0.02
(1.5-47.6) (27.1-73.0) (-48.6 – -2.7)
Error (%) 12.91 24.2 -11.25 p=0.35 p=0.03 p=0.45
(-9.7-35.5) (1.5-46.8) (-33.9-11.4)
Triangle Tracking 2-back Accuracy (%) -28.33 -33.33 5 p<0.001 p<0.001 p=0.76
(-45.5 – -11.2) (-50.5 – -16.2) (-12.1 – 22.1)
Number & Position # of correct responses -12.5 -14.2 1.66 p=0.02 p<0.001 p=0.92
(-23.5 – -1.7)  (-24.9 – -3.4) (-9.1 – 12.4)


Table 5: Computer generated test scores for stroke patients, healthy older adults, and healthy young adults reporting mean differences with 95% CI.


Suggested Citation


Citation: Jinal V, Rini V, Victor P, Sara W, Tanvi B (2017) Test-Retest Reliability of a Computerized Neuropsychological Test Battery: A Cross-Sectional Study Assessing Cognition in Healthy Young and Old Adults, and Stroke Survivors. J Psych Cogn Behav 2017: J107.

Leave a Reply