Development of a Computerized Adaptive Testing System of the Fugl-Meyer Motor Scale in Stroke Patients

doi:10.1016/j.apmr.2011.12.005

Archives of Physical Medicine and Rehabilitation

Volume 93, Issue 6, June 2012, Pages 1014-1020

https://doi.org/10.1016/j.apmr.2011.12.005 Get rights and content

Abstract

Hou W-H, Shih C-L, Chou Y-T, Sheu C-F, Lin J-H, Wu H-C, Hsueh I-P, Hsieh C-L. Development of a computerized adaptive testing system of the Fugl-Meyer motor scale in stroke patients.

Objective

To develop a computerized adaptive testing system of the Fugl-Meyer motor scale (CAT-FM) to efficiently and reliably assess motor function in patients with stroke.

Design

First, a simulation study was used to develop and examine the psychometric properties of the CAT-FM. Second, a field study was employed to determine the administration efficiency of the CAT-FM.

Setting

One medical center and 1 teaching hospital.

Participants

Patients' responses (n=301) were used for the simulation study; 49 patients participated in the field study.

Interventions

Not applicable.

Main Outcome Measures

The 2 CAT-FM item banks (upper extremity and lower extremity) include 37 items from the original Fugl-Meyer scale. The reliability, validity, and responsiveness of the CAT-FM were determined by the simulation study.

Results

Two stopping rules (reliability ≥0.9 or an increase of reliability <.01 after testing an item) were used. The simulation study showed that the CAT-FM had high reliability (≥.93 for upper-extremity and lower-extremity subscales) and concurrent validity (Pearson r≥.91 for the upper-extremity and lower-extremity subscales and motor scale). The responsiveness was moderate (standardized response mean for the upper extremity=.67, lower extremity=.79, and motor=.77) for the 226 patients who completed both assessments at 14 and 90 days after stroke. The field study found that, on average, the time needed to administer the CAT-FM was 242 seconds with 4.7 items.

Conclusions

The CAT-FM is an efficient, reliable, valid, and responsive clinical tool for assessing motor function in patients with stroke.

Section snippets

Methods

The study consisted of 2 phases: (1) a simulation study, developing the CAT-FM and examining the psychometric properties (ie, Rasch reliability, concurrent validity, and responsiveness) of the CAT-FM, and (2) a field study, investigating the administration efficiency (ie, time and number of items needed to administer) and concurrent validity of the CAT-FM. This study protocol was approved by the ethics committee at the participating hospital; all participants gave their informed consent before

Characteristics of the Participants

The characteristics of the 301 patients (177 men and 124 women, with an average age of 62y) in the simulation study recruited at 14 days after stroke are reported in table 1. The scores of the 50-item FM ranged from 2 to 100, showing a wide spectrum of motor function. A total of 75 patients (24.9%) were lost to the follow-up evaluation at 90 days after stroke, but those patients were not significantly different from those who completed both evaluations in terms of age, sex, and motor function.

Discussion

The results showed that the CAT-FM achieved good Rasch reliability (coefficients≥.91), concurrent validity (Pearson correlation coefficients≥.84), responsiveness (SRM≥.67), and efficiency (an average number of 4.7 items and average administration time of 242s) for the motor function assessment in patients with stroke. These findings suggest that the CAT-FM is capable of providing reliable, valid, responsive, and efficient estimates of motor function for patients with stroke.

The CAT-FM appears

Conclusions

The current study showed that the CAT-FM is reliable, valid, responsive, and efficient for assessing motor function in patients with stroke. The CAT-FM is recommended for assessing motor function in patients with stroke in both research and clinical settings.

Supplier

a
The MathWorks, Inc, 3 Apple Hill Dr, Natick, MA 01760.

References (32)

S.M. Haley et al.
Measurement precision and efficiency of multidimensional computer adaptive testing of physical functioning using the pediatric evaluation of disability inventory
Arch Phys Med Rehabil
(2006)
M.L. Woodbury et al.
Longitudinal stability of the Fugl-Meyer Assessment of the upper extremity
Arch Phys Med Rehabil
(2008)
D.L. Hart et al.
Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments
J Clin Epidemiol
(2005)
P.W. Duncan et al.
Defining post-stroke recovery: implications for design and interpretation of drug trials
Neuropharmacology
(2000)
J.A. Husted et al.
Methods for assessing responsiveness: a critical review and recommendations
J Clin Epidemiol
(2000)
F. Malouin et al.
Evaluating motor recovery early after stroke: comparison of the Fugl-Meyer Assessment and the Motor Assessment Scale
Arch Phys Med Rehabil
(1994)
J. Walker et al.
Development of symptom assessments utilising item response theory and computer-adaptive testing–a practical method based on a systematic review
Crit Rev Oncol Hematol
(2010)
D.J. Gladstone et al.
The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties
Neurorehabil Neural Repair
(2002)
C.H. Wang et al.
Inter-rater reliability and validity of the stroke rehabilitation assessment of movement (STREAM) instrument
J Rehabil Med
(2002)
A.R. Fugl-Meyer et al.
The post-stroke hemiplegic patient1. A method for evaluation of physical performance
Scand J Rehabil Med
(1975)

I.P. Hsueh et al.

Psychometric comparisons of 2 versions of the Fugl-Meyer Motor Scale and 2 versions of the stroke rehabilitation assessment of movement

Neurorehabil Neural Repair

(2008)

Y.W. Hsieh et al.

Development and validation of a short form of the Fugl-Meyer motor scale in patients with stroke

Stroke

(2007)

Y.C. Wang et al.

Clinical interpretation of a lower-extremity functional scale-derived computerized adaptive test

Phys Ther

(2009)

I.P. Hsueh et al.

A simplified stroke rehabilitation assessment of movement instrument

Phys Ther

(2006)

I.P. Hsueh et al.

Development of a computerized adaptive test for assessing balance function in patients with stroke

Phys Ther

(2010)

A.M. Jette et al.

Contemporary measurement techniques for rehabilitation outcomes assessment

J Rehabil Med

(2005)

Cited by (31)

Development of a 13-item Short Form for Fugl-Meyer Assessment of Upper Extremity Scale Using a Machine Learning Approach
2023, Archives of Physical Medicine and Rehabilitation
To develop and validate a short form of the Fugl-Meyer Assessment of Upper Extremity Scale (FMA-UE) using a machine learning approach (FMA-UE-ML). In addition, scores of items not included in the FMA-UE-ML were predicted.
Secondary data from a previous study, which assessed individuals post-stroke using the FMA-UE at 4 time points: 5-30 days post-stroke screen, 2-month post-stroke baseline assessment, 6-month post-stroke assessment, and 12-month post-stroke assessment.
Rehabilitation units in hospitals.
A total of 408 individuals post-stroke (N=408).
Not applicable.
The 30-item FMA-UE.
We established 29 candidate versions of the FMA-UE-ML with different numbers of items, from 1 to 29, and examined their concurrent validity and responsiveness. We found that the responsiveness of the candidate versions obviously declined when the number of items was less than 13. Thus, the 13-item version was selected as the FMA-UE-ML. The concurrent validity was good (intra-class correlation coefficients ≥0.99). The standardized response means of the FMA-UE-ML and FMA-UE were 0.54-0.88 and 0.52-0.91, respectively. The Pearson's rs between the change scores of the FMA-UE-ML and those of the FMA-UE were 0.96-0.98. The predicted item scores had acceptable to good accuracy (Kappa=0.50-0.92).
The FMA-UE-ML seems a promising short form to improve administrative efficiency while retaining good concurrent validity and responsiveness. In addition, the FAM-UE-ML can provide all item scores of the FMA-UE for users.
Reducing the Number of Test Items of the Action Research Arm Test Poststroke: A Decision Tree Analysis<sup>✰</sup>
2022, Archives of Physical Medicine and Rehabilitation
The present study aimed to create a shorter version of the Action Research Arm Test (ARAT) without compromising its measurement properties.
Secondary analysis of stroke recovery cohorts that used the ARAT to measure upper limb impairment.
Rehabilitation centers.
Patients with stroke from 5 different stroke recovery cohorts (N=1425).
Not applicable.
A decision tree version of the ARAT (ARAT-DT) was developed using chi-square automated interaction detection. In an independent validation subset, criterion validity, agreement of ARAT-DT with original ARAT scores and score categories, and construct validity with the Fugl-Meyer Upper Extremity Scale score were determined.
In total, 3738 ARAT measurements were available involving 1425 subjects. Chi-square automated interaction detection analysis in the development subset (n=2803) revealed an optimized decision tree with a maximum of 4 consecutive items. In the validation data set (n=935), the ARAT-DT differed by a mean of 0.19 points (0.3% of the total scale) from the original ARAT scores (limits of agreement=−5.67 to 6.05). The ARAT-DT demonstrated excellent criterion validity with the original ARAT scores (intraclass correlation coefficient=0.99 and ρ=0.99) and scoring categories (κ_w=0.97). The ARAT-DT showed very good construct validity with the Fugl-Meyer Upper Extremity Scale (ρ=0.92).
A decision tree version of the ARAT was developed, reducing the maximum number of items necessary for ARAT administration from 19 to 4. The scores produced by the decision tree had excellent criterion validity with original ARAT scores.
Using Machine Learning to Develop a Short-Form Measure Assessing 5 Functions in Patients With Stroke
2022, Archives of Physical Medicine and Rehabilitation
This study aimed to develop and validate a machine learning-based short measure to assess 5 functions (the ML-5F) (activities of daily living [ADL], balance, upper extremity [UE] and lower extremity [LE] motor function, and mobility) in patients with stroke.
Secondary data from a previous study. A follow-up study assessed patients with stroke using the Barthel Index (BI), Postural Assessment Scale for Stroke (PASS), and Stroke Rehabilitation Assessment of Movement (STREAM) at hospital admission and discharge.
A rehabilitation unit in a medical center.
Patients (N=307) with stroke.
Not applicable.
The BI, PASS, and STREAM.
A machine learning algorithm, Extreme Gradient Boosting, was used to select 15 items from the BI, PASS, and STREAM, and transformed the raw scores of the selected items into the scores of the ML-5F. The ML-5F demonstrated good concurrent validity (Pearson's r, 0.88-0.98) and responsiveness (standardized response mean, 0.28-1.01).
The ML-5F comprises only 15 items but demonstrates sufficient concurrent validity and responsiveness to assess ADL, balance, UE and LE functions, and mobility in patients with stroke. The ML-5F shows great potential as an efficient outcome measure in clinical settings.
Effectiveness of a self-rehabilitation program to improve upper-extremity function after stroke in developing countries: A randomized controlled trial
2021, Annals of Physical and Rehabilitation Medicine
Citation Excerpt :
living in or around the city of Cotonou, and; with a Fugl-Meyer Assessment – Upper Extremity (FMA-UE) score 9 to 57 [12]. Patients were excluded if they had:
About two-thirds of stroke patients present long-term upper-limb impairment and limitations of activity, which constitutes a challenge in rehabilitation. This situation is particularly true in developing countries, where there is a need for inexpensive rehabilitation solutions.
This study assessed the effectiveness of a self-rehabilitation program including uni- or bi-manual functional exercises for improving upper-limb function after stroke with respect to the context in Benin, West Africa.
In this single-blind randomized controlled trial, chronic stroke individuals (>6 months post-stroke) performed a supervised home-based self-rehabilitation program for 8 weeks (intervention group); the control group did not receive any treatment. Participants were assessed before treatment (T0), at the end of treatment (T1) and 8 weeks after the end of treatment (T2). The primary outcome was the manual ability of the upper limb, assessed with ABILHAND-Stroke Benin. Secondary outcomes were grip force, motor impairment (Fugl-Meyer Assessment – Upper Extremity), gross manual ability (Box and Block test, Wolf Motor Function test) and quality of life (WHOQOL-26).
We included 28 individuals in the intervention group and 31 in the control group. Adherence to the program was 83%. After 8 weeks of self-rehabilitation, individuals in the intervention group showed significantly improved manual ability and grip force as compared with the control group (P < 0.001), with effect size 0.75 and 0.24, respectively. In the intervention group, the difference in average scores was 10% between T0 and T1 and between T0 and T2. Subscores of physical and psychological quality of life were also significantly improved in the intervention group. The other variables remained unchanged.
A self-rehabilitation program was effective in improving manual ability, grip force and quality of life in individuals with stroke in Benin. More studies are needed to confirm these results in different contexts.
Effectiveness of upper-limb robotic-assisted therapy in the early rehabilitation phase after stroke: A single-blind, randomised, controlled trial
2019, Annals of Physical and Rehabilitation Medicine
Upper-limb robotic-assisted therapy (RAT) is promising for stroke rehabilitation, particularly in the early phase. When RAT is provided as partial substitution of conventional therapy, it is expected to be at least as effective or might be more effective than conventional therapy. Assessments have usually been restricted to the first 2 domains of the International classification of functioning, disability and health (ICF).
This was a pragmatic, multicentric, single-blind, randomized controlled trial to evaluate the effectiveness of upper-limb RAT used as partial substitution to conventional therapy in the early phase of stroke rehabilitation, following the 3 ICF domains.
We randomized 45 patients with acute stroke into 2 groups (conventional therapy, n = 22, and RAT, n = 23). Both interventions were dose-matched regarding treatment duration and lasted 9 weeks. The conventional therapy group followed a standard rehabilitation. In the RAT group, 4 sessions of conventional therapy (25%) were substituted by RAT each week. RAT consisted of moving the paretic upper limb along a reference trajectory while the robot provided assistance as needed. A blinded assessor evaluated participants before, just after the intervention and 6 months post-stroke, according to the ICF domains UL motor impairments, activity limitations, and social participation restriction.
In total, 28 individuals were assessed after the intervention. The following were more improved in the RAT than conventional therapy group at 6 months post-stroke: gross manual dexterity (Box and Block test +7.7 blocks; P = 0.02), upper-limb ability during functional tasks (Wolf Motor Function test +12%; P = 0.02) and patient social participation (Stroke Impact Scale +18%; P = 0.01). Participants’ abilities to perform manual activities and activities of daily living improved similarly in both groups.
For the same duration of daily rehabilitation, RAT combined with conventional therapy during the early rehabilitation phase after stroke is more effective than conventional therapy alone to improve gross manual dexterity, upper-limb ability during functional tasks and patient social participation.
Development of a Computerized Adaptive Testing System for Assessing 5 Functions in Patients with Stroke: A Simulation and Validation Study
2019, Archives of Physical Medicine and Rehabilitation
The authors aimed to develop and validate the Computerized Adaptive Testing System for Assessing 5 Functions in Patients with Stroke (CAT-5F) based on the Barthel Index (BI), Postural Assessment Scale for Stroke patients (PASS), and Stroke Rehabilitation Assessment of Movement (STREAM) to improve the efficiency of assessment. The purposes of the CAT-5F assessment are to describe patients’ levels of impairments or disabilities in the 5 functions and to serve as an outcome measure in patients with stroke.
This is a data-mining study based on data from a previous study using simulation analysis to develop and validate the CAT-5F.
One rehabilitation unit in a medical center in Taiwan served as the setting for this study.
Data were retrieved from totals of 540 (initial assessment) and 309 (discharge assessment) participants with stroke assessed in a previous study. The assessment data (N=540) were from the BI, PASS, and STREAM.
Not applicable.
The outcome measures for this study were from BI, PASS, and STREAM.
The CAT-5F using the optimal stopping rule (limited reliability increased <0.010) had good Rasch reliability across the 5 functions (0.86-0.96) and needed 12.7 items, on average, for the whole administration. The concurrent validity (Pearson product-moment correlation coefficient, r=0.91-0.96) and responsiveness (standardized response mean=0.33-0.91) of the CAT-5F were sufficient in the patients.
The CAT-5F has sufficient administrative efficiency, reliability, concurrent validity, and responsiveness to simultaneously assess basic activities of daily living, postural control, upper extremity/lower extremity motor functions, and mobility in patients with stroke.

View all citing articles on Scopus

: Supported by the National Science Council (grant no. NSC 96-2314-B-037-028), the National Health Research Institutes (grant no. NHRI-EX99-9512PI), and the E-Da Hospital (grant no. EDAHP100002).

: No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated.

: In-press corrected proof published online on Mar 22, 2012, at www.archives-pmr.org.

View full text

Original articleDevelopment of a Computerized Adaptive Testing System of the Fugl-Meyer Motor Scale in Stroke Patients

Abstract

Objective

Design

Setting

Participants

Interventions

Main Outcome Measures

Results

Conclusions

Section snippets

Methods

Characteristics of the Participants

Discussion

Conclusions

Supplier

Arch Phys Med Rehabil

Arch Phys Med Rehabil

J Clin Epidemiol

Neuropharmacology

J Clin Epidemiol

Arch Phys Med Rehabil

Crit Rev Oncol Hematol

The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties

Neurorehabil Neural Repair

Inter-rater reliability and validity of the stroke rehabilitation assessment of movement (STREAM) instrument

J Rehabil Med

The post-stroke hemiplegic patient1. A method for evaluation of physical performance

Scand J Rehabil Med

Psychometric comparisons of 2 versions of the Fugl-Meyer Motor Scale and 2 versions of the stroke rehabilitation assessment of movement

Neurorehabil Neural Repair

Development and validation of a short form of the Fugl-Meyer motor scale in patients with stroke

Stroke

Clinical interpretation of a lower-extremity functional scale-derived computerized adaptive test

Phys Ther

A simplified stroke rehabilitation assessment of movement instrument

Phys Ther

Development of a computerized adaptive test for assessing balance function in patients with stroke

Phys Ther

Contemporary measurement techniques for rehabilitation outcomes assessment

J Rehabil Med

Original article
Development of a Computerized Adaptive Testing System of the Fugl-Meyer Motor Scale in Stroke Patients