Research


Introduction

The ability to move is essential for human behaviour. Disorders that impair motor function have considerable impact on our mobility, manual function and communication, which in turn may influence our ability to perform self-care, work, recreate, and ultimately affect our vitality. The prevalence of disorders that affect motor function, including neurovascular disease (e.g., stroke), neurodegenerative disease (e.g., Parkinson’s disease) and musculoskeletal pain conditions (e.g., osteoarthritis, low back pain) is high and will further increase as a result of an ageing society, on account of the fact that the incidence of these disorders increases with age. This, in combination with the foreseeable future shortage of health care supply, raises the need for effective means to diagnose, monitor and treat disorders associated with motor dysfunction.

Currently, a uniform evaluation of motor function that is efficient, patient-friendly, cost-effective, and that can be carried out on a large scale is not available: every relevant medical discipline uses its own methods. A uniform evaluation is essential in the study and treatment of motor disorders: combined with information on the integrity and functional integration of the musculoskeletal system, the nervous system, and the senses under fluctuating environmental conditions, it will lead to understanding of how different disorders affect motor function. Emerging ICT technologies offer the required breakthrough opportunities to address the shortcomings of traditional evaluations of motor dysfunction.

Project aim

The aim of this project is to develop innovative ICT technologies that facilitate uniform evaluation of motor function and to embed these technologies in medical practice, both in diagnosis and treatment. Specifically, we will (1) use affordable RGB-D cameras (such as Microsoft KinectTM and modern computer-vision techniques to perform accurate, markerless tracking of the motion of patients, (2) develop machine-learning techniques that model the motion dynamics to construct a motion phenogram that captures all key properties of these dynamics and (3) develop virtual and augmented reality techniques for examination of motor function during systematic manipulation of task- and environmental conditions in order to challenge the patient’s adaptive ability, which may provide a more sensitive indicator of problems experienced in daily life.

These innovative technologies will be used to examine movements of the upper extremity (i.e., the arm, hand and fingers) as well as gait and balance. Findings will be compared with current clinical tests. In doing so, we aim to characterize each individual patient by means of a ‘motion phenogram’ and to relate unobtrusively assessed characteristics of motor performance to contributing motor, sensory, and cognitive factors.

Overview of research projects

Upper extremity movements

  • Maximum reachable workspace will be quantified as a measure of upper limb functional capability.
  • Goal-directed movements. Participants have to reach towards a target object presented on a LED screen placed in front them, which may require an overhand or underhand movement. They are instructed to initiate the movement as soon as the target appears, and to reach the target as quickly and as accurately as possible. The positioning and appearance of target objects will be varied systematically to not only examine the execution of goal directed movements, but also other characteristics of the diseases under study, e.g., neglect in stroke patients or impulsive behavior in PD patients.
  • Reaching and grasping will be evaluated using augmented reality techniques. Augmented Reality (AR) will be exploited to assess the patient’s ability to adequately adapt their grasping behavior to objects of different sizes and shapes. Moreover, the functional integration of systems involved in motor control (i.e., motor, sensory and cognitive) can be evaluated in various contexts using systematic manipulations of the visual environment. Task difficulty will be gradually increased by varying the size, shape and position of the virtual objects and their target positions. To what extent is a patient able to adequately correct an ongoing movement when an obstacle suddenly appears in the motion path? And how robust is the performance when a cognitive dual task is administered or when static and dynamic distractors are presented?

Gait and balance

  • Unperturbed walking on a 10-meter walkway at a self-selected, comfortable walking speed while various gait parameters (such as walking speed, cadence, step length and step symmetry) are collected using the Interactive Walkway (IWW) setup.
  • Gait adaptability tests. The IWW is used to implement movement-dependent augmented reality by projecting, for example, visual stepping targets, obstacles and barriers onto the walking surface with a projector. The augmented visual context allows for assessing various aspects of gait adaptability and gait-environment interactions, including targeted stepping, sudden stopping, time-pressured turning, sudden obstacle negotiation and walking at an imposed speed.
  • Turning is an essential test in patients with PD and especially useful in patients with ‘freezing’. Using the IWW, motor performance will be quantified when patients (attempt to) rise from a chair, walk to a line that is 3 meters away, turn (180°), walk back and sit down again (the so-called ‘timed up-and-go’ test).
  • Balance tests including normal stance with feet 15-20 cm apart, followed by stance with feet closed, tandem stance and standing on one leg, all performed first with eyes open and then with eyes closed, will be performed on an instrumented treadmill so that displacements of the center of pressure can be recorded.

With these setups for assessment of “upper extremity movements” and “gait and balance” the TIM project aims to further develop low cost setups for unobtrusive, sensitive, reliable and valid quantification of motor (dys)function and contributing motor, sensory and/or cognitive factors. Efficient, unobtrusive, and patient-friendly assessment setups may help physicians and physiotherapists to characterize an individual’s movement pattern, select the optimal strategy for the treatment of a specific patient, and to monitor changes in motor function over time or in response to the selected treatment.

Related research

logo_neurocontrol

medical-delta-research.original    propark_logo

Output

Datasets can be found at the Downloads page.

Publications

2019

  • Daphne Geerse (2019) ‘A comprehensive approach to assess walking ability and fall risk using the Interactive Walkway’ (thesis)
  • Geerse DJ, Roerdink M, Marinus J, van Hilten JJ. Walking adaptability for targeted fall-risk assessments. Gait Posture. 2019;70:203-210.

2018

  • Geerse, D.J., Roerdink, M., Marinus, J., van Hilten, J.J. Assessing walking adaptability in Parkinson’s disease: “ the Interactive Walkway”. Front Neurol 2018;9:1096.
  • Bank, P.J.M., Cidota, M.A., Ouwehand, P.W., Lukosch, S.G. Patient-tailored augmented reality games for assessing upper extremity motor impairments in Parkinson’s disease and stroke. J Med Systems, 42:246. doi: 10.1007/s10916-018-1100-9
  • Bank, P.J.M., Marinus, J., van Tol, R.M., Groeneveld, I.F., Goossens, P.H., de Groot, J.H., van Hilten, J.J., Meskers, C.G.M. Cognitive-motor interference in during goal-directed upper-limb movements. Eur J Neurosci, 48:3146-3158. doi: 10.1111/ejn.14168
  • Pintea, S.L., Zheng, J., Li, X., Bank, P.J.M., van Hilten, J.J., van Gemert, J.C. Hand-tremor frequency estimation in videos. ECCV-2018 Workshop on observing and understanding hands in action, Munich (GE), 2018. Received the ‘Best paper Award’.

2017

  • Geerse, D., Coolen, B., Kolijn, D., Roerdink, M. Validation of Foot Placement Locations from Ankle Data of a Kinect v2 Sensor. Sensors (Basel) 2017;17(10). pii: E2301. doi: 10.3390/s17102301.
  • Bank, P.J.M., Marinus, J., Meskers, C.G.M., de Groot, J.H., van Hilten, J.J. Optical hand tracking: a novel technique for the assessment of bradykinesia in Parkinson’s Disease. Movement Disorders Clinical Practice, doi: 10.1002/mdc3.12536
  • Bank, P.J.M.,  Dobbe, L.R.M., Meskers, C.G.M., de Groot, J.H., de Vlugt, E. Manipulation of visual information affects control strategy in visuomotor tracking. Behavioral Brain Research, 329, 205-214.
  • Geerse, D.J., Coolen, B.H., Roerdink, M. Walking-adaptability assessments with the Interactive Walkway: Between-systems agreement and sensitivity to task and subject variations. Gait Posture 2017;54:194-201. doi: 10.1016/j.gaitpost.2017.02.021
  • Geerse, D.J., Coolen, H., Kolijn, D.T.T.I., Roerdink, M. Validation of Foot Placement Locations from Ankle Data of a Kinect v2 Sensor. Sensors 2017, 17, 2301. doi:10.3390/s17102301

2016

  • Cidota, M.A., Clifford, R.M.S., Lukosch, S.G., Billinghurst, M. “Using visual effects to facilitate depth perception for spatial tasks in virtual and augmented reality”. IEEE International Symposium on Mixed and Augmented Reality Adjunct Proceedings, pp. 172-177.
  • Cidota, M.A., Lukosch, S.G., Dezentje, P., Bank, P.J.M., Lukosch, H.K., Clifford, R.M.S. “Serious Gaming in Augmented Reality using HMDs for Assessment of Upper Extremity Motor Dysfunctions – User studies for Engagement and Usability”. Accepted for publication (June 2016) in i-com: Journal of Interactive Media (http://www.degruyter.com/view/j/icom), Special Issue on Smartglass Technologies, Applications and Experiences.
  • Kooij, J.F.P., Van Gemert, J. C. Depth-aware Motion Magnification. In Proceedings of the European Conference on Computer Vision, pp. 467-482,. Springer International Publishing. doi:10.1007/978-3-319-46484-8_28
  • Kooij, J.F.P. SenseCap: synchronized data collection with Microsoft Kinect2 and LeapMotion. Proceedings of the 2016 ACM on Multimedia Conference. p. 1218-1221. ACM New York, NY, USA. doi:10.1145/2964284.2973805
  • van der Meulen, E., Cidota, M.A., Lukosch, S.G., Bank, P.J.M., van der Helm, A.J.C., Visch, V. “A Haptic Serious Augmented Reality Game for Motor Assessment of Parkinson’s Disease Patients”.  IEEE International Symposium on Mixed and Augmented Reality Adjunct Proceedings, pp. 102-104.
  • Geerse, D.J., Roerdink, M., Coolen, B., Marinus, J., van Hilten, J.J. The interactive walkway: Towards assessing gait-environment interactions in a clinical setting [abstract]. Mov Disord. 2016; 31 (suppl 2).
  • Bank, P.J.M., Marinus, J., de Groot, J.H., van Hilten, J.J., Meskers, C.G.M. Assessment of cognitive-motor interference during an upper extremity motor task in Parkinson’s disease [abstract]. Mov Disord. 2016; 31 (suppl 2).
  • Bank, P.J.M., Marinus, J., de Groot, J.H., Meskers, C.G.M., van Hilten, J.J. Objective, unobtrusive assessment of bradykinesia in Parkinson’s disease [abstract]. Mov Disord. 2016; 31 (suppl 2).

2015

  • Cidota, M.A., Clifford, R.M.S., Dezentje, P., Lukosch, S.G., Bank, P.J.M. “[POSTER] Affording Visual Feedback for Natural Hand Interaction in AR to Assess Upper Extremity Motor Dysfunction”, IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Fukuoka, 2015, pp. 92-95, doi: 10.1109/ISMAR.2015.29
  • Cidota, M.A., Lukosch, S.G., Bank, P.J.M. “Augmented reality for motion analysis of patients with upper extremity motor dysfunction”, VAAT, 2015, 3rd IEEE VR International Workshop on Virtual and Augmented Assistive Technology (VAAT) 2015, pp. 1-4, doi:10.1109/VAAT.2015.7155401
  • Dezentje, P., Cidota, M.A., Clifford, R.M.S., Lukosch S.G., Bank, P.J.M., Lukosch, H.K. “Designing for Engagement in Augmented Reality Games to Assess Upper Extremity Motor Dysfunctions”, IEEE International Symposium on Mixed and Augmented Reality – Media, Art, Social Science, Humanities and Design (ISMAR-MASH’D), Fukuoka, 2015, pp. 57-58, doi: 10.1109/ISMAR-MASHD.2015.24
  • Geerse, D.J., Coolen, H., Roerdink, M. Kinematic validation of a multi-Kinect V2 instrumented 10-meter walkway for quantitative gait analysis. PLoS One 2015;10(10):e0139913.

Oral presentations

2019

  • Daphne Geerse. Walking adaptability for targeted fall-risk assessments. ISPGR congress. Edinburgh, United Kingdom, 30 June-4 July 2019.

2017

  • Daphne Geerse. Validation of foot placement locations and step lengths on the Interactive Walkway. ISPGR congress. Fort Lauderdale, USA, 25-29 June 2017.

2016

  • Linda Bank. Assessment of cognitive-motor interference during an upper extremity motor task in Parkinson’s Disease. NeuroCIMT-Neurocontrol symposium. Egmond aan Zee, NL, 24 May 2016.
  • Julian F. P. Kooij. Synchronized Data Collection with Microsoft Kinect2 and LeapMotion. Open Source Software Competition at ACM Multimedia. Amsterdam, NL, 17-19 October 2016.

2015

  • Linda Bank. Manipulation of visual information to assess adaptability of motor control (NEURAS-V). 5th Dutch Conference on Bio-Medical Engineering. Egmond aan Zee, NL, 22-23 January 2015.
  • Linda Bank. NEURAS1: Integrative assessment of motor (dys)function. Neurocontrol-Neurosipe symposium. Soesterberg, NL, 13-15 April 2015.
  • Daphne Geerse. The Interactive Walkway: Towards assessing gait-environment interactions in a clinical setting. ISPGR conference. Seville, SP, 2 July 2015.

Poster presentations

2019

  • Daphne Geerse. Walking adaptability for targeted fall-risk assessments. ISPGR congress. Edinburgh, United Kingdom, 30 June-4 July 2019.

2017

  • Geerse, D.J., Roerdink, M., Coolen, H., Ouwehand, P.W., Marinus, J., van Hilten, J.J. The sudden stop-and-start test of the Interactive Walkway affords an innovative evaluation of freezing of gait in Parkinson’s disease patients. ISPGR congress. Fort Lauderdale, USA, 25-29 June 2017.
  • Geerse, D.J., Coolen, H., Roerdink, M. Walking-adaptability assessments with the Interactive Walkway: Between-systems agreement and sensitivity to task and subject variations. ISPGR congress. Fort Lauderdale, USA, 25-29 June 2017.
  • Geerse, D.J., Coolen, H., Roerdink, M. Walking-adaptability assessments with the Interactive Walkway: Between-systems agreement and sensitivity to task and subject variations. Science Exchange Day. Amsterdam, the Netherlands, 29 September 2017.
  • Geerse, D.J., Roerdink, M., Coolen, H., Ouwehand, P.W., Marinus, J., van Hilten, J.J. Fall risk assessments with the Interactive Walkway. SMALLL congress. Leuven, Belgium, 17 November 2017.
  • Geerse, D.J., Roerdink, M., Coolen, H., Ouwehand, P.W., Marinus, J., van Hilten, J.J. Fall risk assessments with the Interactive Walkway. VvBN PhD-day. Rotterdam, the Netherlands, 1 December 2017.
  • Geerse, D.J., Roerdink, M., Coolen, H., Ouwehand, P.W., Marinus, J., van Hilten, J.J. Fall risk assessments with the Interactive Walkway. ParkinsonNet congress. Rotterdam, the Netherlands, 7 December 2017.

2016

  • Bank, P.J.M., Marinus, J., de Groot, J.H., Meskers, C.G.M.,  van Hilten, J.J. Objective, unobtrusive assessment of bradykinesia in Parkinson’s Disease. NeuroCIMT-Neurocontrol symposium. Egmond aan Zee, NL, 23-24 May 2016.
  • Bank, P.J.M., Marinus, J., de Groot, J.H., van Hilten, J.J., Meskers, C.G.M.  Assessment of cognitive-motor interference during an upper extremity motor task in Parkinson’s Disease. NeuroCIMT-Neurocontrol symposium. Egmond aan Zee, NL, 23-24 May 2016.
  • Geerse, D.J., Roerdink, M., Coolen, B., Marinus, J., van Hilten, J.J. The interactive walkway: Towards assessing gait-environment interactions in a clinical setting. 20th International Congress of Parkinson’s Disease and Movement Disorders. Berlin, DE, 22 June 2016.
  • Bank, P.J.M., Marinus, J., de Groot, J.H., Meskers, C.G.M.,  van Hilten, J.J. Objective, unobtrusive assessment of bradykinesia in Parkinson’s Disease. 20th International Congress of Parkinson’s Disease and Movement Disorders. Berlin, DE, 22 June 2016.
  • Bank, P.J.M., Marinus, J., de Groot, J.H., van Hilten, J.J., Meskers, C.G.M.  Assessment of cognitive-motor interference during an upper extremity motor task in Parkinson’s Disease. 20th International Congress of Parkinson’s Disease and Movement Disorders. Berlin, DE, 22 June 2016.
  • Kooij, J.F.P., Van Gemert, J. C. Depth-aware Motion Magnification. European Conference on Computer Vision (ECCV). Amsterdam, NL, 11-14 October 2016.
  • Geerse, D.J., Roerdink, M., Coolen, B., Marinus, J., van Hilten, J.J. The interactive walkway: Towards assessing gait-environment interactions in a clinical setting. SMALLL conference. Enschede, NL, 2 December 2016.

2015

  • Bank, P.J.M., Marinus, J., de Groot, J.H., van Hilten, J.J., Meskers, C.G.M. Integrated assessment of motor (dys)function in stroke and Parkinson’s disease patients (NEURAS-IMA). 5th Dutch Conference on Bio-Medical Engineering. Egmond aan Zee, NL, 22-23 January 2015.
  • Bank, P.J.M., Marinus, J., de Groot, J.H., van Hilten, J.J., Meskers, C.G.M. Integrated assessment of motor (dys)function in stroke and Parkinson’s disease patients (NEURAS-IMA). Neurocontrol-Neurosipe symposium. Soesterberg, NL, 13-15 April 2015.
  • Bank, P.J.M., Marinus, J., de Groot, J.H., van Hilten, J.J., Meskers, C.G.M. Integrated assessment of motor (dys)function in stroke and Parkinson’s disease patients (NEURAS-IMA). Congress on Neurorehabilitation and Neural Repair. Maastricht, NL, 21-22 May 2015.

2014

  • Daphne Geerse, Bert Coolen & Melvyn Roerdink. The Interactive Walkway: Hello world! Assessing gait-environment interactions in a clinical setting. SMALLL conference. Nijmegen, NL, 28 November 2014.

Other

2019

  • Roerdink, M., Geerse, D.J., Coolen, H. Mixed reality for posture and gait research: principles and applications (workshop). ISPGR congress. Edinburgh, United Kingdom, 30 June-4 July 2019.

2017

  • Roerdink, M., Timmermans, C., Geerse, D.J., Coolen, H. Movement-dependent event control: principles and applications (workshop). ISPGR congress. Fort Lauderdale, USA, 25-29 June 2017.

2016

  • Stand at “ParkinsonNet congres”. Rotterdam, NL, 8 December 2016.

2015

  • Participation in “LUMC Wetenschapsdag”. Leiden, NL, 25 Oktober 2015.