Human Motion Capture and Identification for Assistive Systems Design in Rehabilitation. Pubudu N. Pathirana

is commonly utilised to assist patients in recovery and reacquire ADLs.

      Defined as “the treatment of disease, injury, or deformity by physical methods such as massage, heat treatment, and exercise rather than by drugs or surgery” [279], physiotherapy (also known as physical therapy) has been applied in clinics for thousands of years. A number of therapies are included in physical therapy, such as mechanotherapy, hydrotherapy, balneotherapy and so on [349], among which mechanotherapy was documented as early as the 1840s. In recent decades, physical therapy has been applied extensively for various musculoskeletal injuries and neurological movement disorders [107, 142]. The detailed examples can be found in Section 1.2.

      Although traditional physical therapy has shown its effectiveness for the rehabilitation of physical functions of patients with movement disorders [95], a series of drawbacks can also be observed [9, 57], which are summarised as follows.

       These rehabilitation programmes are “boring” and patients are demotivated by these repetitive exercises.

       Computerised sensing techniques are not involved in these programmes, which may lead to incorrect interpretation of observed data.

       A one‐to‐one form of delivering rehabilitation services makes conventional rehabilitation very inefficient and costly.

       Costly equipment is required by traditional rehabilitation therapies.

       Insufficient funding for rehabilitation services results in making access to these services unaffordable.

       The workforce in the rehabilitation field is inadequate in number.

       Rehabilitation centres are usually distributed in urban areas, while a large number of people needing rehabilitation services live in rural regions.

      In light of the above, in 1998, the term “telerehabilitation” was “first raised” in a scientific article, attempting to overcome the shortage in conventional rehabilitation services [303]. Co‐existing with the opportunities for telerehabilitation, such as economy of scale, interactive and motivation, reduced healthcare costs, patients' privacy prevention and so on [57], a number of challenges can be seen from an engineering point of view as well. These challenges are also closely related to the aim of this book.

      First of all, developing affordable, high‐quality and robust hardware [300] is critical to capture human movements for further analysis. High‐quality and effective hardware may provide more accurate monitoring of the movements, thereby offering better feedback to patients to correct their movements and more valuable information to therapists to make further treatment decisions. In addition, similar to the fourth issue in traditional rehabilitation mentioned above, costly equipment used in telerehabilitation services may prevent patients with low economic status from accessing them. Thus, how to develop affordable devices is critical for the development of telerehabilitation services.

Secondly, an advanced approach in representing human movement data [381] is important after capturing human motion. As pointed out in Theodoros and Russell [350], one difficulty in telerehabilitation is how to reduce information collected from sensors, thereby producing meaningful results to therapists. Therefore, an emerging challenge is to discover which features can be utilised to represent human motions so that their rehabilitation‐related details can be well preserved while other irrelevant information can be reduced as much as possible.

      Thirdly, developing an outcome measurement scheme to quantitatively and objectively represent the performance of patients accessing telerehabilitation services in also a challenge [382]. As previously described [10], one of the goals of RERC is to develop assessment tools to monitor the progress of patients accessing telerehabilitation services. These tools can not only be a feedback to stimulate the patients to perform more exercises but can also provide therapists with general information about their patients in terms of functional rehabilitation.

      Last but not the least, enabling patients to access physical telerehabilitation services regardless of their location and time is a challenge [382]. As one purpose of rehabilitation services is for patients to recover the ability to perform ADLs, enabling them to perform rehabilitation exercises in their most familiar and natural environment is very important. Due to the different preferences between patients, developing telerehabilitation services that can be run on mobile devices is extremely useful. By doing so, patients' kinematic performance in telerehabilitation exercises and daily living can be assessed pervasively.

      Because of the importance of telerehabilitation, as well as the automated kinematic performance assessment tool, a significant amount of effort has been made to improve the physical telerehabilitation, which will be discussed in Sections 1.5 and 1.6.

1.3 Sensors in Telerehabilitation

      In the past few decades, various types of sensors have been considered as patient monitoring and data acquisition tools. In this section, the use of three main types of sensors, namely Kinect, RGB camera and IMU, is reviewed.

      1.3.1 Opto‐electronic sensing

      The Vicon [7] motion capture studio is used in some clinical settings as well as in certain other human motion assessment applications in sports. A number of fixed cameras within a dedicated infrastructure are used to capture the position of markers on the moving body part to greater precision, with the resulting Vicon system being used as the benchmark for motion capture systems. Due to cost, the requirement for a fixed and dedicated infrastructure and the specialised technical knowledge, this has not been used as a standard clinical practice and remains predominantly used in research and development.

      Recently, a number of non‐invasive, portable and affordable optical 3D motion capture devices have emerged. These products include Leap Motion® controller, ASUS®Xtion PRO LIVE, Intel®Creative Senz3D, Microsoft Kinect®, and so on. Among them, Kinect is the most popular motion capture device for whole-body motion capture. The first version was released in 2010 with Xbox 360 and was used for gaming purposes and the second version was released with Xbox One in 2014.

The first version of Kinect utilised a depth sensor provided by a company named “PrimeSense” [400]. The appearance and components of this version of Kinect are shown in Figure 1.3.

Figure 1.3 Appearance and components of Kinect version 1. Source: Evan‐Amos, Image taken from https://commons.wikimedia.org/w/index.php?curid=16059165.

Figure 1.4 The pinhole camera model of Kinect version 1 [373]. Source: From Wang et al. [373]. © 2012, University of British Columbia.

The infrared projector and the corresponding camera can be modelled as in Figure 1.4.

These sensors measure the depth information via a structured light principle, which analyses a pattern (such as that in Figure 1.5) of bright spots projected on to the surface of an object [328]. In the case of Kinect, these “bright spots”