Multimodal Control System of Active Lower Limb Exoskeleton with Feedback

Biofeedback is crucial for controlling human movement during walking [1]. Sensory function disorders resulting from injuries and diseases lead to a loss of movement ability, as well as motor function impairments. An issue of organizing feedback in a pilot-exoskeleton system in the process of developing robotic lower limb medical exoskeleton often is not considered. It is assumed that when exoskeleton pilot walks, retained sensory function is sufficient to organize movement, i.e. lost motor function is replaced by the exoskeleton function, but lost sensory function is not replaced.

A number of research projects devoted to the development of robotic lower limb exoskeleton interfaces attempted to organize feedback in a biomechanical pilot-exoskeleton system by means of vibrostimulation, acoustic signals, visual indicators [1], and electrostimulation [2]. However, despite the proven feasibility of real-time feedback on the pressure exerted on the foot by the surface [2], or on the current angles of joints, feedback systems did not find a practical application in the production of robotic lower limb exoskeletons. The main reason is inability to organize a direct channel for proportional control of exoskeleton drive for the pilots who lost both sensory and motor lower limb function (who are considered the main target audience for lower limb robotic exoskeletons). Without a direct proportional control, feedback channel does not provide any advantages. Direct proportional control of exoskeleton drive was achieved only in the exoskeleton HAL, which is based on multichannel EMG signals, registered on the lower limbs muscles of a pilot. Thus, such technology could not be used for pilots who lost both sensory and motor lower limb function. At the same time, command control systems have received wide application in mass-produced robotic lower limb exoskeletons [3‐6]. By performing a certain action, for example by pressing a button on crutches, an operator informs the exoskeleton control system of his/her intention to take a step with the left foot. By performing another action, for example, by pressing another button, an operator can communicate his/her intention to take a step with the right foot. This approach allows the system to produce a movement of the pilot-exoskeleton system on a flat surface and does not require significant time spent on pilot training. Moreover, command control does not require significant concentration of a pilot, which conserves pilot’s energy and has a positive effect on the duration of continuous walking. The main disadvantage of command control systems is the difficulty to adapt to changing environment (slopes, obstacles, doorstep, etc.). Lower limb movement in such conditions is executed according to a pre-pattern and, at best, can switch from one template to another during the walking mode.

Thus, we can state that a promising robotic lower limb medical exoskeleton must support at least two modes of operation:

The latter should be implemented using feedback, allowing the pilot with impaired lower limb sensory function sense exoskeleton current status and orientation of its parts relative to each other.

Switching between modes in a multimode robotic exoskeleton should be executed in the process of movement, without stopping or any special procedures.

This work is devoted to developing and testing a multimode active control system of the lower limb exoskeleton with feedback.

Active lower limb exoskeleton with four degrees of freedom developed in the Lobachevsky State University of Nizhni Novgorod was used as the basis for of the multimodal robotic exoskeleton. Active movement is executed in the sagittal plane with knee and hip joints. The exoskeleton is driven by brushless electric motors Maxon EC 45 flat 36 V (knee joints) and Maxon EC 90 flat 36 V (hip joints). Actuators are controlled by a group of 4 controllers Electriprivod BLSD-20 (Russia). Controllers are calibrated in torque units (Nm), which is exerted by the actuators of exoskeleton joints. Drive units of right and left limbs with spur gearboxes provide a torque of 100 Nm in the knee joint and 150 Nm in the hip joint. The ankle is passive. A 36 V, 10 Ah lithium polymer battery provides 2 h of continuous motion of the pilot-exoskeleton system. Exoskeleton is attached to the pilot’s torso via a segmented belt and to the limbs via U-shaped staples.

Robotic lower limb exoskeleton sensory subsystem created in the framework of this study is based on the sensory subsystem described in [7]. Four force sensors were added to the original subsystem and placed on the soles of the feet (2 sensors on each foot). Tekscan FlexiForce A401 strain gauges were used as force sensors to provide an ability to measure the reaction force on the soles of the feet in the range of 0–1220 N at frequency of 100 Hz.

Given the improvements, exoskeleton sensory subsystem provides the following abilities:

Data acquisition from sensors and issuing control commands to exoskeleton actuators is executed on board the microprocessor board BeagleBoard-xM rev. C [8]. Sampling frequency of data acquisition is 100 Hz.

In order to construct human–exoskeleton interface with feedback, we proposed a scheme that uses strain gauges placed on the phalanges of the exoskeleton pilot. Strain gauges provide establishment of the exoskeleton proportional drive control channel. It took eight strain gauges in total, four on each of the pilots’ wrist, to control four actuators, each of which can rotate the shaft in two opposite directions.

In addition to strain gauges on the phalanges of the pilots’ fingers, we placed buttons that provide an ability to promptly change the operational mode of exoskeleton control system. Location of sensors and buttons is illustrated in Fig. 1.

To provide feedback, we decided to use visual stream from a display unit attached to the pilots’ head and placed in his field of vision. The following information should be displayed on the unit block:

A general view of a visual feedback display unit and its attachment method to the exoskeleton pilots’ head are shown in Fig. 2.

Display unit is operated by a microcontroller, which executes the following tasks:

Interaction of BeagleBoard-xM microprocessor board with the display unit is executed via RS485 interface by means of RS232-RS485 converter.

A scheme of developed human–machine interface is shown in Fig. 3.

Control system of exoskeleton supports six modes: STANDBY, BLOCKAGE, STAND UP, SIT DOWN, AUTO MOVEMENT and MANUAL MOVEMENT. Modes could be switched with non-locking buttons placed near the base of the index fingers of exoskeleton pilot. Button placed on the right hand is referred to as the SWR and button placed on the left hand is called SWL. Buttons are pressed with the thumbs. Let us review the operational modes:

STANDBY. Exoskeleton control system goes into this mode immediately after switching on. Voltage to actuators is not supplied, only sensors are surveyed and exoskeleton status is displayed on a display unit (symbol ‘F’ is displayed on a single-character indicator). Holding either SWR or SWL buttons will switch exoskeleton control system to STANDBY mode from any other mode.

BLOCKAGE. This mode is activated with a single press of either SWR or SWL buttons. In this mode, a small current flows through the motor winding, which blocks motor shafts and does not allow exoskeleton joints to flex or stretch. All the sensors are surveyed and exoskeleton status is displayed on a display unit (symbol ‘L’ is displayed on a single-character indicator).

STAND UP. This mode is activated by double pressing SWR button if pilot-exoskeleton system is currently in a sitting position. In this mode, control system executes a template movement to bring exoskeleton in a standing position. All the sensors are surveyed and exoskeleton status is displayed on a display unit (symbol ‘U’ is displayed on a single-character indicator). Upon completion of stand up command, BLOCKAGE mode is automatically activated.

SIT DOWN. This mode is activated by pressing the SWR button three times given that pilot-exoskeleton system is in a standing position. In this mode, control system executes a template movement to bring exoskeleton in a sitting position. All the sensors are surveyed and exoskeleton status is displayed on a display unit (symbol ‘S’ is displayed on a single-character indicator). Upon completion of this procedure, STANDBY mode is automatically activated.

AUTO MOVEMENT. This mode is activated by double pressing SWL button if pilot-exoskeleton system is currently in a standing position. In this mode, the control system monitors the deviation of the pilot torso from the vertical in frontal and sagittal planes, measures forces exerted on the exoskeleton feet and flexing angles of the joints. Exoskeleton status is displayed on the display unit (symbol ‘A’ is displayed on a single-character indicator). If sensors indicate that one of the legs is both a support leg and is in front of the other leg, a step forward is made with the other leg. This way, a pilot with impaired motor and sensory lower limb functions can walk helping himself with his hands relying on the crutches. In the AUTO MOVEMENT mode, movement can only be executed on a flat surface with a slope of 15 degrees or less.

MANUAL MOVEMENT. This mode is activated by pressing SWL button three times. In this mode exoskeleton control system polls all sensors and displays exoskeleton status on a display unit (symbol ‘H’ is displayed on a single-character indicator). Each strain gauge is associated with an exoskeleton joint and with the direction of motor shaft rotation. For example, a strain gauge mounted on the distal phalange of the index finger is associated with hip flexion, while a strain gauge mounted on the medial phalange of the same finger is associated with hip straightening. Movement of the knee joint is controlled by the strain gauges mounted to phalanges of the middle finger in a similar way to the index finger.

Described interface has been implemented and tested at National Research N.I. Lobachevsky State University of Nizhni Novgorod. The test results confirmed the possibility of creating a multimodal active control system of lower limbs exoskeleton with feedback, which allows prompt adapting of the exoskeleton movement to the environment in the process of movement.

On the basis of this research, we sent an application for invention titled “Methods for generating control signals and manual control of lower limb exoskeleton operation, as well as control interfaces for operation of this exoskeleton in manual and software control modes using this generation method”, registered by the Federal Service for Intellectual Property of Russian Federation #2016144426 11.11.2016.

The challenge of developing a control interface for an active lower limb exoskeleton that is easy to use for pilots with significant impairments of sensory and motor lower limb functions is one of the main problems preventing active lower limb exoskeleton from widespread application in the medical field.

Solution to this problem lies in implementing proportional control and feedback mechanisms that would allow using the human brain plasticity for replacing lost sensory and motor lower limb functions.

Proposed interface has significant potential to improve consumer properties in terms of improving proportional control channel and feedback channel. In the near future we plan to replace bulky display unit with augmented reality glasses and implement feedback channel by means of cutaneous vibratory stimulation.