Download PDF
Methods Inf Med 2017; 56(02): 95-111
DOI: 10.3414/ME16-02-0013
REHAB
Schattauer GmbH

Combined Vision and Wearable Sensors-based System for Movement Analysis in Rehabilitation

Sofija Spasojević
1   School of Electrical Engineering, University of Belgrade, Belgrade, Serbia
2   Mihailo Pupin Institute, University of Belgrade, Belgrade, Serbia
3   Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal
,
Tihomir V. Ilić
4   Department of Neurophysiology, Medical Faculty of Military Medical Academy, University of Defense, Belgrade, Serbia
,
Slađan Milanović
5   Institute for Medical Research, Department of Neurophysiology, University of Belgrade, Serbia
,
Veljko Potkonjak
1   School of Electrical Engineering, University of Belgrade, Belgrade, Serbia
,
Aleksandar Rodić
2   Mihailo Pupin Institute, University of Belgrade, Belgrade, Serbia
,
José Santos-Victor
3   Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal
› Author Affiliations Funding: This work is partially funded by the Ministry of Education, Science and Technology Development of the Republic of Serbia under the contracts TR-35003, III-44008, III-44004 and #ON175012. This work was partially funded by the EU Project POETICON++ and the Portuguese FCT Project [UID/EEA/50009/2013]. The work is complementary supported by the Alexander von Humboldt project ..Emotionally Intelligent Robots – EIrobots”, Contract no. 3.4-IP-DEU/112623.
Further Information

Publication History

received: 06 March 2016

accepted: 22 October 2016

Publication Date:
25 January 2018 (online)

    Permissions and Reprints

Summary

Background: Traditional rehabilitation sessions are often a slow, tedious, disempowering and non-motivational process, supported by clinical assessment tools, i.e. evaluation scales that are prone to subjective rating and imprecise interpretation of patient’s performance. Poor patient motivation and insufficient accuracy are thus critical factors that can be improved by new sensing/processing technologies.

Objectives: We aim to develop a portable and affordable system, suitable for home rehabilitation, which combines vision-based and wearable sensors. We introduce a novel approach for examining and characterizing the rehabilitation movements, using quantitative descriptors. We propose new Movement Performance Indicators (MPIs) that are extracted directly from sensor data and quantify the symmetry, velocity, and acceleration of the movement of different body/hand parts, and that can potentially be used by therapists for diagnosis and progress assessment.

Methods: First, a set of rehabilitation exercises is defined, with the supervision of neurologists and therapists for the specific case of Parkinson’s disease. It comprises full-body movements measured with a Kinect device and fine hand movements, acquired with a data glove. Then, the sensor data is used to compute 25 Movement Performance Indicators, to assist the diagnosis and progress monitoring (assessing the disease stage) in Parkinson’s disease. A kinematic hand model is developed for data verification and as an additional resource for extracting supplementary movement information.

Results: Our results show that the proposed Movement Performance Indicators are relevant for the Parkinson’s disease assessment. This is further confirmed by correlation of the proposed indicators with clinical tapping test and UPDRS clinical scale. Classification results showed the potential of these indicators to discriminate between the patients and controls, as well as between the stages that characterize the evolution of the disease.

Conclusions: The proposed sensor system, along with the developed approach for rehabilitation movement analysis have a significant potential to support and advance traditional rehabilitation therapy. The main impact of our work is two-fold: (i) the proposition of an approach for supporting the therapists during the diagnosis and monitoring evaluations by reducing subjectivity and imprecision, and (ii) offering the possibility of the system to be used at home for rehabilitation exercises in between sessions with doctors and therapists.

Keywords

Rehabilitation - movement analysis - Kinect - wearable sensors
 

  • References

  • 1 Jankovic J. Parkinson’s disease: clinical features and diagnosis. J Neurol, Neurosurgery and Psychiatry. 2008; 79 (04) 368-376.
    PubMedGoogle Scholar
  • 2 Goetz C, Poewe W, Rascol O. et al. Movement Disorder Society Task Force report on the Hoehn and Yahr staging scale: Status and Recommendations. Mov Disord. 2004; 19 (09) 1020-1028.
    CrossrefPubMedGoogle Scholar
  • 3 Goetz C, Tilley B, Shaftman S. et al. Movement Disorder Society-Sponsored Revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): Scale Presentation and Clinimetric Testing Results. Mov Disord. 2008; 23 (15) 2129-2170.
    CrossrefPubMedGoogle Scholar
  • 4 Stamford JA, Schmidt PN, Friedl KE. What Engineering Technology Could Do for Quality of Life in Parkinson’s Disease: A Review of Current Needs and Opportunities. IEEE J Biomed Health Inform. 2015; 19 (06) 1862-1872.
    CrossrefPubMedGoogle Scholar
  • 5 Zhou H, Hu H. Human motion tracking for rehabilitation – A survey. Biomed Signal Process Control. 2008; 3 (01) 1-18.
    CrossrefPubMedGoogle Scholar
  • 6 Parisi F, Ferrari G, Giuberti M. et al. Body-Sensor-Network-Based Kinematic Characterization and Comparative Outlook of UPDRS Scoring in Leg Agility, Sit-to-Stand, and Gait Tasks in Parkinson’s Disease. IEEE J Biomed Health Inform. 2015; 19 (06) 1777-1793.
    CrossrefPubMedGoogle Scholar
  • 7 Patel S, Park H, Bonato P. et al. A review of wearable sensors and systems with application in rehabilitation. J Neuroeng Rehabil. 2012; 9: 21.
    CrossrefPubMedGoogle Scholar
  • 8 Gonzalez-Jorge H, Riveiro B, Vazquez-Fernandez E. et al. Metrological evaluation of Microsoft Kinect and Asus Xtion sensors. Measurement. 2013; 46 (06) 1800-1806.
    CrossrefPubMedGoogle Scholar
  • 9 Gonçalves AR, Gouveia ER, Cameirão MS. et. al. Automating senior fitness testing through gesture detection with depth sensors. IET International Conference on Technologies for Active and Assisted Living (TechAAL). 2015. Nov. 5; London, UK.:
    Google Scholar
  • 10 Antón D, Goñi A, Illarramendi A. Exercise Recognition for Kinect-based Telerehabilitation. Methods Inf Med. 2015; 54 (02) 145-155.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 11 Khoshelham K, Elberink S. Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors. 2012; 12 (02) 1437-1454.
    CrossrefPubMedGoogle Scholar
  • 12 Clark R, Pu Y, Fortina K. et al. Validity of the Microsoft Kinect for assessment of postural control. Gait and Posture. 2012; 36 (03) 372-377.
    CrossrefPubMedGoogle Scholar
  • 13 Chang C, Lange B, Zhang M. et al. Towards Pervasive Physical Rehabilitation Using Microsoft Kinect. 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth). 2012. May 21–24; San Diego, CA.: 159-162.
    Google Scholar
  • 14 Chang Y, Han W, Tsai Y. A Kinect-based upper limb rehabilitation system to assist people with cerebral palsy. Res Dev Disabil. 2013; 34 (11) 3654-3659.
    CrossrefPubMedGoogle Scholar
  • 15 Chang Y, Chen S, Huang J. A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Res Dev Disabil. 2011; 32 (06) 2566-2570.
    CrossrefPubMedGoogle Scholar
  • 16 Gama A, Chaves T, Figueiredo L. et al. Guidance and Movement Correction Based on Therapeutics Movements for Motor Rehabilitation Support Systems. 14th Symposium on Virtual and Augmented Reality (SVR); 2012 May 28–31. Rio de Janiero, Brasil.: IEEE; 2012: 191-200.
    Google Scholar
  • 17 Calin A, Cantea A, Dascalu A. et al. Mira – Upper Limb Rehabilitation System using Microsoft Kinect. Studia Universitatis Babes-Bolyai, Informatica. 2011; 56 (04) 63-74.
    PubMedGoogle Scholar
  • 18 Galna B, Barry G, Jackson D. et al. Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson’s disease. Gait and Posture. 2014; 39 (04) 1062-1068.
    CrossrefPubMedGoogle Scholar
  • 19 Galna B, Jackson D, Schofield G. et al. Retraining function in people with Parkinson’s disease using the Microsoft Kinect: game design and pilot testing. J Neuroeng Rehabil. 2014: 11-60.
    PubMedGoogle Scholar
  • 20 Albiol-Pérez S, Lozano-Quilis J, Gil-Gómez H. et al. Virtual rehabilitation system for people with Parkinson’s disease. 9th international conference on disability, virtual reality and associated technologies (ICDVRAT). 2012. Sept. 10–12; Laval, France.: 423-426.
    Google Scholar
  • 21 Spasojevic S, Santos-Victor J, Ilic T. et al. A Vision-based System for Movement Analysis in Medical Applications: the example of Parkinson’s Disease. Nalpantidis L, Krüger V, Eklundh JO, Gasteratos A. Proceedings of the 10th International Conference on Computer Vision Systems – Volume 9163 (ICVS 2015).. New York: Springer; 2015: 424-434.
    Google Scholar
  • 22 Okuno R, Yokoe M, Akazawa K. et al. Finger Taps Movement Acceleration Measurement System for Quantitative Diagnosis of Parkinson’s disease. 28th Annual International Conference of the Engineering in Medicine and Biology Society (EMBS). 2006. Aug. 30 – Sept. 3; New York, NY.: IEEE; 2006: 6623-6626.
    Google Scholar
  • 23 Smeja M, Foerster F, Fuchs G. et al. 24-h Assessment of tremor activity and posture in Parkinson’s disease by multi-channel accelerometry. J Psycho-physiol. 1999; 13 (04) 245-256.
    CrossrefPubMedGoogle Scholar
  • 24 LeMoyne R, Coroian C, Mastroianni T. Quantification of Parkinson’s disease characteristics using wireless accelerometers. International Conference on Complex Medical Engineering (ICME). 2009. April 9–11; Tempe, AZ.: IEEE; 2009: 1-5.
    Google Scholar
  • 25 Salarian A, Russmann H, Wider C. et al. Quantification of tremor and bradykinesia in Parkinson’s disease using a novel ambulatory monitoring system. IEEE Trans Biomed Eng. 2007; 54 (02) 313-322.
    CrossrefPubMedGoogle Scholar
  • 26 Burkhard PR, Langston JW, Tetrud JW. Voluntarily simulated tremor in normal subjects. Clin Neurophysiol. 2002; 32 (02) 119-126.
    CrossrefPubMedGoogle Scholar
  • 27 Kandori A, Yokoe M, Sakoda S. et al. Quantitative magnetic detection of finger movements in patients with Parkinson’s disease. Neurosci Res. 2004; 49 (02) 253-260.
    CrossrefPubMedGoogle Scholar
  • 28 Shima K, Tsuji T, Kan E. et al. Measurement and Evaluation of Finger Tapping Movements Using Magnetic Sensors. 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS). 2008 Aug. 20–25; Vancouver, BC.: IEEE; 2008: 5628-5631.
    Google Scholar
  • 29 Shima K, Tsuji T, Kandori A. et al. Measurement and Evaluation of Finger Tapping Movements Using Log-linearized Gaussian Mixture Networks. Sensors. 2009; 9 (03) 2187-2201.
    CrossrefPubMedGoogle Scholar
  • 30 Niazmand K, Tonn K, Kalaras A. et al. Quantitative Evaluation of Parkinson’s Disease using sensor based smart Glove. 24th International Symposium on Computer-Based Medical Systems (CBMS). 2011 June 27–30; Bristol, UK.: IEEE; 2011: 1-8.
    Google Scholar
  • 31 Prochazka A, Bennett DJ, Stephens MJ. et al. Measurement of Rigidity in Parkinson’s Disease. Mov Disord. 1997; 12 (01) 24-32.
    CrossrefPubMedGoogle Scholar
  • 32 Hanson M, Powell H, Frysinger R. et al. Teager Energy Assessment of Tremor Severity in Clinical Application of Wearable Inertial Sensors. IEEE/ NIH Life Science Systems and Applications Workshop (LISA). 2007 Nov. 8–9; Bethesda, MD.: IEEE; 2007: 136-139.
    Google Scholar
  • 33 Iacono RP, Lonser RR, Maeda G. et al. Chronic Anterior Pallidal Stimulation for Parkinson’s Disease. Acta Neurochir. 1995; 137 1-2 106-112.
    CrossrefPubMedGoogle Scholar
  • 34 Su Y, Geng D, Allen CR. et al. Three-Dimensional Motion System (“Data-Gloves”): Application for Parkinson’s disease and Essential Tremor. IEEE International Workshop on Virtual and Intelligent Measurement Systems (VIMS). 2001 May 19–20; Budapest, Hungary.: IEEE; 2001: 28-33.
    Google Scholar
  • 35 Su Y, Allen CR, Geng D. et al. 3-D Motion System (“Data-Gloves”): Application for Parkinson’s disease. IEEE Trans Instrum Meas. 2003; 52 (03) 662-674.
    CrossrefPubMedGoogle Scholar
  • 36 Morrow K, Docan C, Burdea G. et al. Low-cost Virtual Rehabilitation of the Hand for Patients Post-Stroke. International IEEE Workshop on Virtual Rehabilitation 2006. New York, NY.: IEEE; 2006: 6-10.
    Google Scholar
  • 37 Lange B, Koenig S, McConnell E. et al. Interactive game-based rehabilitation using the Microsoft Kinect. IEEE Virtual Reality Short Papers and Posters (VRW); 2012 March 4–8. Costa Mesa, CA.: IEEE; 2012: 171-172.
    Google Scholar
  • 38 Cho CW, Chao WH, Lin SH. et al. A vision-based analysis system for gait recognition in patients with Parkinson’s disease. Expert Syst Appl. 2009; 36 (03) 7033-7039.
    CrossrefPubMedGoogle Scholar
  • 39 Lum P, Burgar C, Shor P. et al. Robot-Assisted Movement Training Compared With Conventional Therapy Techniques for the Rehabilitation of Upper-Limb Motor Function After Stroke. Arch Phys Med Rehabil. 2002; 83 (07) 952-959.
    CrossrefPubMedGoogle Scholar
  • 40 Vaisman L, Dipietro L, Krebs H. A Comparative Analysis of Speed Profile Models for Wrist Pointing Movements. IEEE Trans Neural Syst Rehabil Eng. 2013; 21 (05) 756-766.
    CrossrefPubMedGoogle Scholar
  • 41 Plamondon R. A kinematic theory of rapid human movements. Part I. Movement representation and generation. Biol Cybern. 1995; 72 (04) 295-307.
    CrossrefPubMedGoogle Scholar
  • 42 Gribble P, Ostry D. Origins of the Power Law Relations Between Movement Velocity and Curvature: Modeling the Effects of Muscle Mechanics and Limb Dynamics. J Neurophysiol. 1996; 76 (05) 2853-2860.
    CrossrefPubMedGoogle Scholar
  • 43 Bullock D, Grossberg S. Adaptive neural networks for control of movement trajectories invariant under speed and force rescaling. Hum Mov Sci. 1991; 10 (01) 3-53.
    CrossrefPubMedGoogle Scholar
  • 44 Mirkov D, Milanovic S, Ilic D. et al. Symmetry of Discrete and Oscillatory Elbow Movements: Does it Depend on Torque That the Agonist and Antagonist Muscle can Exert?. Motor Control. 2002; 6 (03) 271-281.
    CrossrefPubMedGoogle Scholar
  • 45 Available from: http://www.cyberglovesystems.com/products/cyberglove-II/overview.
    PubMed
  • 46 Maetzler W, Domingos J, Srulijes K. et al. Quantitative Wearable Sensors for Objective Assessment of Parkinson’s Disease. Mov Disord. 2013; 28 (12) 1628-1637.
    CrossrefPubMedGoogle Scholar
  • 47 Keus S, Bloem B, Hendriks E. et al. Evidence-Based Analysis of Physical Therapy in Parkinson’s Disease with Recommendations for Practice and Research. Mov Disord. 2007; 22 (04) 451-460.
    CrossrefPubMedGoogle Scholar
  • 48 Pötter-Nerger M, Wenzelburger R, Deuschl G. et al. Impact of Subthalamic Stimulation and Medication on Proximal and Distal Bradykinesia in Parkinson’s Disease. Eur Neurol. 2009; 62 (02) 114-119.
    CrossrefPubMedGoogle Scholar
  • 49 Okuno R, Yokoe M, Fukawa K. et al. Measurement system of finger-tapping contact force for quantitative diagnosis of Parkinson’s disease. 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS); 2007 Aug. 22–26. Lyon, France.: IEEE; 2007: 1354-1357.
    Google Scholar
  • 50 Available from: http://petercorke.com/Robotics_Toolbox.html.
    PubMed
  • 51 Spong MW, Hutchinson S, Vidyasagar M. Robot Modeling and Control.. John Wiley & Sons; 2006
  • 52 Field A. Discovering statistics using SPSS.. 3rd ed. London: Sage; 2009
  • 53 Fisher RA. The use of multiple measurements in taxonomic problems. Annals of Eugenics. 1936; 7 (02) 179-188.
    CrossrefPubMedGoogle Scholar