Skip to main content
main-content

2022 | Buch

Intelligent Robotics and Applications

15th International Conference, ICIRA 2022, Harbin, China, August 1–3, 2022, Proceedings, Part II

herausgegeben von: Honghai Liu, Zhouping Yin, Prof. Lianqing Liu, Li Jiang, Prof. Guoying Gu, Xinyu Wu, Weihong Ren

Verlag: Springer International Publishing

Buchreihe: Lecture Notes in Computer Science

share
TEILEN
insite
SUCHEN

Über dieses Buch

The 4-volume set LNAI 13455 - 13458 constitutes the proceedings of the 15th International Conference on Intelligent Robotics and Applications, ICIRA 2022, which took place in Harbin China, during August 2022.

The 284 papers included in these proceedings were carefully reviewed and selected from 442 submissions. They were organized in topical sections as follows: Robotics, Mechatronics, Applications, Robotic Machining, Medical Engineering, Soft and Hybrid Robots, Human-robot Collaboration, Machine Intelligence, and Human Robot Interaction.

Inhaltsverzeichnis

Frontmatter

Rehabilitation and Assistive Robotics

Frontmatter
Design and Control of a Bimanual Rehabilitation System for Trunk Impairment Patients

Stroke is one of the leading causes of severe long-term disability worldwide. Past research has proved the effectiveness and importance of trunk rehabilitation in enhancing the locomotive capacities and balance of patients with hemiplegia. The current robot-assisted rehabilitation mainly focuses on limbs but less on trunk training. In this paper, we have developed a bimanual rehabilitation system for trunk impairment patients. The main contribution of the work is twofold. First, the synchronized training trajectory generation method of the two articulated robots is proposed by introducing a simplified yet effective trunk kinematic model and taking the past demonstrations as experience. Second, an improved admittance controller is embedded to modulate the training trajectories online in catering to subjects’ different locomotion capacities. The physical experiments are conducted to validate the feasibility of our system. It is expected that, upon the availability of clinical trials, the effectiveness of our system in the locomotion restoring training of trunk impairment patients will be further confirmed.

Lufeng Chen, Jing Qiu, Lin Zhou, Hongwei Wang, Fangxian Jiang, Hong Cheng
Study on Adaptive Adjustment of Variable Joint Stiffness for a Semi-active Hip Prosthesis

The moment and impedance of the hip antagonist muscle groups can change dynamically with movement, but existing passive hip prostheses are unable to achieve this function and limit the walking ability of hip amputees. This paper presents a novel adaptive adjustment stiffness system for a semi-active hip prosthesis that can simulate the function of hip antagonist muscles according to real-time detection of gait phase information and improve the walking ability of hip amputees. A method for predicting hip prosthesis kinematic information in advance is proposed to improve real-time performance of stiffness adjustment. The kinematic information at the amputee's healthy side is acquired from the posture sensor, and the kinematic information of the prosthetic side is predicted using a nonlinear autoregressive neural network model. To realize adaptive adjustment of variable stiffness joint, we have developed a stiffness controller based on PID algorithm. The target values for stiffness controller are obtained by analyzing the walking of the healthy person, and the actual stiffness of the hip joint on the prosthetic is determined using angle sensors and torque sensors from a customized embedded system. The hip joint stiffness tracking experiments show that the stiffness controller can effectively improve the regularity of the stiffness curve of the prosthetic joint, and the joint torque is significantly increased. The gait symmetry tests show that gait symmetry indices $${\text{SI}}$$ and $${\text{R}}_{{\text{II}}}$$ were improved by 86.3% and 85.1%, respectively. The average difference in bilateral gait length of walking amputees was reduced from 6.6 cm to 0.7 cm. The adjustable stiffness system proposed in this study can adaptively adjust the stiffness of the prosthetic joint to the amputee's gait, effectively improving the amputee's walking ability.

Meng Fan, Yixi Chen, Bingze He, Qiaoling Meng, Hongliu Yu
A Hip Active Lower Limb Support Exoskeleton for Load Bearing Sit-To-Stand Transfer

Sit-to-stand (STS) transfer is a basic and important motion function in daily living. Most currently-existing studies focus on movement assistance for patients who lost mobility or have impaired their muscle strength. In this article, a hip-active lower limb support exoskeleton is designed to assist load bearing STS transfer for healthy persons. In order to provide effective assistance, a self-designed quasi-direct drive actuator is adopted to compose the actuation module and a load bearing stand up assistance strategy is designed based on virtual modwhutel control and gravity compensation. Control parameters are optimized in a musculoskeletal simulation environment with kinematic and kinetic data obtained from the wearer. The experimental results show that muscle activation levels of gluteus maximus and semimembranous are reduced with the help from the proposed exoskeleton during load bearing STS transfer.

Jinlong Zhou, Qiuyan Zeng, Biwei Tang, Jing Luo, Kui Xiang, Muye Pang
Generative Adversarial Network Based Human Movement Distribution Learning for Cable-Driven Rehabilitation Robot

Movement distribution analysis can reveal the body’s changes from training with rehabilitation robotic assistance, and the distribution result has been used to develop robot control scheme. However, movement distribution modeling and further validation of the control scheme remain a problem. In this study, we propose a generative adversarial network (GAN) to learn the distribution of human movement, which will be used to design the control scheme for a cable-driven robot later. We preliminary collect a movement dataset of ten healthy subjects following a circular training trajectory, and develop a GAN model based on WGAN-GP to learn the distribution of the dataset. The distribution of the generated data is close to that of the real dataset (Kullback-Leibler divergence = 0.172). Ergodicity is also used to measure the movement trajectories generated by our GAN model and that of the real dataset, and there is no significant difference (p = 0.342). The results show that the developed GAN model can capture the features of human movement distribution effectively. Future work will focus on conducting further experiments based on the proposed control scheme, integrating human movement distribution into the control of real cable-driven robot, recruiting subjects for robot training experiments and evaluation.

Zonggui Li, Chenglin Xie, Rong Song

Learning from Human Demonstrations for Redundant Manipulator

Frontmatter
Kinematic Analysis and Optimization of a New 2R1T Redundantly Actuated Parallel Manipulator with Two Moving Platforms

Redundantly actuated parallel manipulators (RAPMs) with two rotations and one translation (2R1T) are suitable for some manufacturing applications where high precision and speed are required. In this paper, a new 2R1T RAPM with two output moving platforms, called Var2 RAPM, is proposed. The Var2 RAPM is a (2RPR-R)-RPS-UPS RAPM, which is actuated by four P joints (where R denotes a revolute joint, P denotes an actuated prismatic joint, S denotes a spherical joint, and U denotes a universal joint). First, the mobility analysis shows that the second moving platform can achieve two rotations and one translation. Then, the inverse kinematics is presented by using the constraints of joints. In addition, one local and two global kinematic performance indices are used to evaluate the motion/force transmissibility of the proposed Var2 RAPM. Finally, with respect to the two global kinematic indices, the architectural parameters of Var2 RAPM are optimized by using the parameter-finiteness normalization method. The Var2 RAPM has the potential for the machining of curved workpieces.

Lingmin Xu, Xinxue Chai, Ziying Lin, Ye Ding
Control Design for a Planar 2-DOF Parallel Manipulator: An Active Inference Based Approach

Active inference controller has the advantages of simplicity, high efficiency and low computational complexity. It is based on active inference framework which is prominent in neuroscientific theory of the brain. Although active inference has been successfully applied to neuroscience, its application in robotics are highly limited. In this paper, an active inference controller is adopted to steer a 2-DOF parallel manipulator movement from the initial state to the desired state. Firstly, the active inference controller is introduced. Secondly, Dynamic model of parallel manipulator system with constraints is established by Udwaida-Kalaba equation. Thirdly, apply the active inference controller to the parallel manipulator system. Finally, the simplicity and effectiveness of control effect are verified by numerical simulations.

Duanling Li, Yixin He, Yanzhao Su, Xiaomin Zhao, Jin Huang, Liwei Cheng
Geometrical Parameter Identification for 6-DOF Parallel Platform

The positioning accuracy of the 6-degree-of-freedom (DOF) parallel platform is affected by the model accuracy. In this paper, the parameter identification of the 6-DOF parallel platform is carried out to obtain the accurate kinematic model. The theoretical inverse kinematic model is established by closed-loop vector method. The rotation centers of each spherical joint and hooker joint are selected as geometric parameters to be identified. The inverse kinematic model with geometric error is established. The geometric parameters are identified by iterative least square method. Simulation results show that the parameter identification method is correct.

Jian-feng Lin, Chen-kun Qi, Yan Hu, Song-lin Zhou, Xin Liu, Feng Gao
Multi-robot Cooperation Learning Based on Powell Deep Deterministic Policy Gradient

Model-free deep reinforcement learning algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods could not perform well in multi-agent environments due to the instability of teammates’ strategies. In this paper, a novel reinforcement learning method called Powell Deep Deterministic Policy Gradient (PDDPG) is proposed, which integrates Powell’s unconstrained optimization method and deep deterministic policy gradient. Specifically, each agent is regarded as a one-dimensional variable and the process of multi-robot cooperation learning is corresponding to optimal vector searching. A conjugate direction in Powell-method is constructed and is used to update the policies of agents. Finally, the proposed method is validated in a dogfight-like multi-agent environment. The results suggest that the proposed method outperforms much better than independent Deep Deterministic Policy Gradient (IDDPG), revealing a promising way in realizing high-quality independent learning.

Zongyuan Li, Chuxi Xiao, Ziyi Liu, Xian Guo
Model Construction of Multi-target Grasping Robot Based on Digital Twin

In the unstructured environment, how to grasp the target object accurately and flexibly as human is always a hot spot in the field of robotics research. This paper proposes a multi-object grasping method based on digital twin to solve the problem of multi-object grasping in unstructured environment. The twin model of robot grasping process is constructed to realize multi-object collision-free grasping. Firstly, this paper analyzes the composition of digital twin model based on the five-dimensional digital twin structure model, and constructs the twin model of unknown target grasping process combining with multi-target grasping process of robot. Then, a digital model of the key elements in the multi-target grasping process of the robot is established to realize the interaction between the physical entity and the virtual model, and an evaluation model of the virtual model grasping strategy is established to screen the optimal grasping strategy. Finally, based on the twin model of multi-target grasping process, a multi-target grasping process based on digital twin is constructed to realize the multi-target stable grasping without collision.

Juntong Yun, Ying Liu, Xin Liu

Mechanism Design, Control and Sensing of Soft Robots

Frontmatter
A New Geometric Method for Solving the Inverse Kinematics of Two-Segment Continuum Robot

The inverse kinematics (IK) of continuum robot (CR) is an important factor to guarantee motion accuracy. How to construct a concise IK model is very essential for the real-time control of CR. A new geometric algorithm for solving the IK of CR is proposed in this paper. Based on Piecewise Constant Curvature (PCC) model, the kinematics model of CR is constructed and the envelope surface of the single segment is calculated. The IK of CR is obtained by solving the intersection of surfaces. The spatial problem is transformed into a plane using a set of parallel planes, and the intersection is solved using the k-means clustering algorithm. The algorithm's real-time performance and applicability are improved further by increasing the sampling rate and decreasing the range of included angles. A distinct sequence is designed for solving the IK of CR. The efficiency and effectiveness of geometric are validated in comparison to some of the most popular IK algorithms. Finally, the accuracy of the algorithm is further validated by a physical prototype experiment.

Haoran Wu, Jingjun Yu, Jie Pan, Guanghao Ge, Xu Pei
All-in-One End-Effector Design and Implementation for Robotic Dissection of Poultry Meat

The meat dissection is yet a labor-intensive operation in the poultry industry all around the world, and the technologies making it unmanned are emerging. A low-cost robotic poultry meat dissection processing cycle, and its end-effector, are proposed in this work. An all-in-one end-effector capable of multi-configuration for a series of dissection tasks, e.g., off-shackle, cone-fixture, and butterfly-harvesting, is implemented for a single manipulator. The structural operation environment in the processing line is highly utilized as a priori; that way the demand for complicated perception capability of a robot is significantly reduced. As a whole, the hardware and processing proposed is developed with an in-depth adaptation to the industry demand. An experimental platform testing the robotic poultry dissection was settled up, and the processing cycle and hardware proposed was validated. The manipulator-harvested butterfly obtained in our platform demonstrates a smoothness quality on the breast separation surfaces that is clearly comparable with that of the manually handled one. It turns out that the robotic dissection processing proposed is of the satisfactory in terms of the industrial application demand.

Han Liu, Jingjing Ji
Motor Imagery Intention Recognition Based on Common Spatial Pattern for Manipulator Grasping

With further development of brain-computer interface (BCI) technology and the BCI medicine field, people with movement disorders can be treated by using artificial limbs or some external devices such as an exoskeleton to achieve sports function rehabilitation. At present, brain-computer interfaces (BCI) based on motor imagination mainly have problems such as accuracy that need to be improved. The key to solving such problems is to extract high-quality EEG signal features and reasonably select classification algorithms. In this paper, the efficient decoding of dichotomy EEG intentions is verified by combining features such as Common Spatial Pattern (CSP) with different classifiers in the PhysioNet public data set. The results show that: CSP is used as the feature, and the support vector machine (SVM) has the best classification effect. The highest classification accuracy of five subjects reaches 87% and the average accuracy reaches 80%. At the same time, the CSP-SVM algorithm was used in the qbrobotics manipulator to conduct a grasping experiment to verify the real-time performance and effectiveness of the algorithm. This provides a new solution for the brain-computer interface to control robotic auxiliary equipment, which is expected to improve the daily life of the disabled.

Wenjie Li, Jialu Xu, Xiaoyu Yan, Chengyu Lin, Chenglong Fu
Event-Triggered Secure Tracking Control for Fixed-Wing Aircraft with Measurement Noise Under Sparse Attacks

This study focuses on the secure tracking control problem of fixed-wing aircraft with multiple output sensors under sparse attacks and measurement noise, where the main challenge is to achieve secure state estimation. An output estimation method based on the Grubbs test is proposed to exclude outliers before estimation to improve the estimated accuracy. Moreover, an event-triggered mechanism is designed to transmit data only when more than half of the measurement data meet the triggering conditions, thereby alleviating the interference of sparse attacks with normal triggering. Based on the event-triggered output estimation, an extended high-gain state observer is designed, which ensures the output prediction accuracy by using the reference output, and estimates the matched model uncertainty as an extended state to decrease the estimation errors. Finally, the tracking controller is designed with the estimated states. The control performance of the proposed method is verified by a numerical simulation.

Guangdeng Chen, Xiao-Meng Li, Wenbin Xiao, Hongyi Li
Backstepping Fuzzy Adaptive Control Based on RBFNN for a Redundant Manipulator

Redundant manipulator is a highly nonlinear and strongly coupled system. In practical application, dynamic parameters are difficult to determine due to uncertain loads and external disturbances. These factors will adversely affect the control performance of manipulator. In view of the above problems, this paper proposes a backstepping fuzzy adaptive control algorithm based on the Radial Basis Function Neural Network (RBFNN), which effectively eliminates the influence of the internal uncertainty and external interference on the control of the manipulator. Firstly, the algorithm adopts the backstepping method to design the controller framework. Then, the fuzzy system is used to fit the unknown system dynamics represented by nonlinear function to realize model-free control of the manipulator. The fuzzy constants are optimized by RBFNN to effectively eliminate the control errors caused by unknown parameters and disturbance. Finally, in order to realize RBFNN approximating the optimal fuzzy constant, an adaptive law is designed to obtain the weight value of RBFNN. The stability of the closed-loop system is proved by using Lyapunov stability theorem. Through simulation experiments, the algorithm proposed in this paper can effectively track the target joint angle when the dynamic parameters of the 7-DOF redundant manipulator are uncertain and subject to external torque interference. Compared with fuzzy adaptive control, the tracking error of the algorithm in this paper is smaller, and the performance is better.

Qinlin Yang, Qi Lu, Xiangyun Li, Kang Li
A Flexible Electrostatic Adsorption Suction Cup for Curved Surface

As an important electrical pipeline, the GIS (gas insulated metal-enclosed switchgear) pipeline is applied in many scenarios. However, it has a narrow internal place and non-magnetic wall, which makes existing robot adsorption structure difficult to meet the needs of this type of the pipe. This paper combines the flexible electrostatic adsorption electrode and the flexible material to design a suction cup adapted to curved surface and narrow pipeline inner wall. The relationship between electrostatic adsorption force and the applied voltage is studied through designing a coplanar bipolar electrostatic adsorption membranes. And the adsorption force simulation are completed by the Ansoft Maxwell, the structure of the flexible electrostatic suction cup analyzed by Abaqus. After making the suction cup prototypes with different structures, the experiment testing adsorption force determines the final structure and verifies the adsorption capacity of electrostatic suction cup.

Haotian Guan, Shilun Du, Yaowen Zhang, Wencong Lian, Cong Wang, Yong Lei
SCRMA: Snake-Like Robot Curriculum Rapid Motor Adaptation

Controllers for underwater snake-like robots are difficult to design because of their high DOF and complex motions. Additionally, because of the complex underwater environment and insufficient knowledge of hydrodynamics, the traditional control algorithms based on environment or robot modeling cannot work well. In this paper, we propose an SCRMA algorithm, which combines the characteristics of the RMA algorithm for rapid learning and adaptation to the environment, and uses curriculum learning and save &load exploration to accelerate the training speed. Experiments show that the SCRMA algorithm works better than other kinds of reinforcement learning algorithms nowadays.

Xiangbei Liu, Fengwei Sheng, Xian Guo
Investigation on the Shape Reconstruction of Cable-Driven Continuum Manipulators Considering Super-Large Deflections and Variable Structures

The Cable-driven continuum manipulator is a kind of robot with high flexibility and dexterity. It has been attracting extensive attention in the past few years due to its potential applications for complex tasks in confined spaces. However, it is a challenge to accurately obtain the manipulator’s shape, especially for super-large deflections, like more than 180 $$^{\circ }$$ . In this paper, a novel shape reconstruction method based on Bézier curves is proposed for cable-driven continuum manipulators considering super-large defections and variable structures. In addition, a variety of structures of continuum manipulators are considered, where five kinds of manipulators with variable cross sections are manufactured and tested. The feasibility of the proposed method is verified by both simulations and experiments. Results show that the shape reconstruction errors (root-mean-square-error) are all less than 2.35 mm, compared to the 180 mm long manipulators, under various manipulator structures.

Yicheng Dai, Xiran Li, Xin Wang, Han Yuan

Intelligent Perception and Control of Rehabilitation Systems

Frontmatter
An Improved Point-to-Feature Recognition Algorithm for 3D Vision Detection

Vision-detection-based grasping is one of the research hotspots in the field of automated production. As the grasping scenes become more and more diversified, 3D images are increasingly chosen as the input images for object recognition in complex recognition scenes because they can describe the morphology and pose information of the scene target objects more effectively. With object recognition and pose estimation in 3D vision as the core, this paper proposes an improved pose estimation algorithm based on the PPF feature voting principle for the problems of low recognition rate and poor real-time performance in vision detection systems. The algorithm firstly performs preprocessing measures such as voxel downsampling and normal vector calculation on the original point cloud to optimize the point cloud quality and reduce the interference of irrelevant data. Secondly, an improved point cloud downsampling strategy is proposed in the point cloud preprocessing stage, which can better preserve the surface shape features of the point cloud and avoid introducing a large number of similar surface points. Finally, an improved measure of scene voting ball is proposed in the online recognition stage. The recognition and matching experiments on the public dataset show that the proposed algorithm has an average recognition rate improvement of at least 0.2%.

Jianyong Li, Qimeng Guo, Ge Gao, Shaoyang Tang, Guanbo Min, Chengbei Li, Hongnian Yu
Characteristic Analysis of a Variable Stiffness Actuator Based on a Rocker-Linked Epicyclic Gear Train

This paper presents the characteristic analysis and mechanical realization of a novel variable stiffness actuator based on a rocker-linked epicyclic gear train (REGT-VSA). The stiffness adjustment of the actuator works by converting the differential motion of the planetary gear train into the linear motion of the elastic element. The unique design of the rocker-linked epicyclic gear train ensures excellent compactness and easy controllability, which enables the actuator to be qualified for constructing a manipulator toward cooperation applications. However, the output position and stiffness of the actuator may be affected by the mechanism clearance. The paper introduces characteristic analysis of stiffness and clearance, and carries out a series of related simulations. The analysis results can provide guidelines for the high-quality assembly of lever-based VSA.

Zhisen Li, Hailin Huang, Peng Xu, Yinghao Ning, Bing Li
Facial Gesture Controled Low-Cost Meal Assistance Manipulator System with Real-Time Food Detection

In view of the increasing number of people with independent eating disorders in the world, the need to design and develop a low-cost and effective meal assistance manipulator system has become more and more urgent. In this paper, we propose a low-cost meal assistance manipulator system that uses facial gesture based user intent recognition to control the manipulator, and integrates real-time food recognition and tracking to achieve different food grabbing. The system is divided into three stages. The first stage is real-time food detection and tracking based on yolov5 and deepsort, which completes the classification, positioning and tracking of food and pre-selects the target food. The second stage is user intent recognition based on facial gesture. The user selects the target food through the change of facial gesture and controls the feeding timing. The third stage is the grabbing and feeding based on image servo. Finally, four volunteers tested the system to verify the effectiveness of the system. See videos of our experiments here: https://www.bilibili.com/video/BV1Z54y1f78E/

Xiao Huang, LongBin Wang, Xuan Fan, Peng Zhao, KangLong Ji
Improved Cascade Active Disturbance Rejection Control for Functional Electrical Stimulation Based Wrist Tremor Suppression System Considering the Effect of Output Noise

The wrist tremor suppression system designed based on functional electrical stimulation technology has been welcomed by the majority of tremor patients as a non-invasive rehabilitation therapy. Due to the complex physiological structure characteristics of the wrist musculoskeletal system, it is difficult to accurately model it, and the measurement noise at the output port of the wrist tremor suppression system is difficult to avoid. These problems seriously affect the performance of tremor suppression. In order to solve the above problems, an improved linear active disturbance rejection control (LADRC) scheme is proposed based on the cascade strategy. The simulation results show that the proposed improved cascade LADRC can not only meet the requirements of system control performance against model uncertainty, but also attenuate the influence of output noise on the system, so as to effectively improve the suppression performance of tremor.

Changchun Tao, Zan Zhang, Benyan Huo, Yanhong Liu, Jianyong Li, Hongnian Yu
An Anti-sideslip Path Tracking Control Method of Wheeled Mobile Robots

Anti-sideslip has not been paid much attention by most researchers of wheeled mobile robots. And some existing anti-sideslip path tracking control methods based on switching control have problems such as relying on design experience. To enable the wheeled mobile robot to prevent sideslip and track the reference path at the same time, we propose an anti-sideslip path tracking control method based on a time-varying local model. The principle of this method is to make model predictions and rolling optimizations in the robot coordinate system in each control period. The proposed controller is tested by MATLAB simulation. According to the simulation results, the proposed controller can prevent sideslip when the wheeled mobile robot tracks the reference path. Even if the ground adhesion coefficient is low, the maximum lateral speed of the robot is only 0.2159 m/s. While preventing sideslip, the proposed controller is able to keep the displacement error of path tracking within 0.1681 m. Under the same conditions, the maximum absolute value of the displacement error of the proposed controller is at least 55.15% smaller than that of the controller based on the global model.

Guoxing Bai, Yu Meng, Qing Gu, Guodong Wang, Guoxin Dong, Lei Zhou
Design of a Practical System Based on Muscle Synergy Analysis and FES Rehabilitation for Stroke Patients

In the diagnosis and rehabilitation of motor function for stroke patients, the combination of motor function assessment based on Muscle Synergy Analysis (MSA) and rehabilitation using Functional Electrical Stimulation (FES) is a new strategy, which has been validated its feasibility and superiority in clinical rehabilitation. However, it is difficult to be extended to a larger patient population because of low equipment integration, high cost, and complicated operation. This paper designed a hardware and software integrated system for MSA and FES, to achieve functional integration, portability, and simplicity of operation. The hardware system implements multi-channel sEMG acquisition and FES. The software system achieves device control, data processing, and algorithm analysis with a simple and clear user interface. The functions of the system were preliminarily validated by the data of healthy people. This system solves the current problems of equipment function separation and complicated data processing. It realizes the integration of diagnosis and rehabilitation processes, and helps to promote the further development and application of stroke intelligent rehabilitation system.

Ruikai Cao, Yixuan Sheng, Jia Zeng, Honghai Liu
Adaptive Force Field Research on Plane Arbitrary Training Trajectory of Upper Limb Rehabilitation Robot Based on Admittance Control

In order to further improve the flexibility, safety and training pertinence of the desktop end-traction type upper limb rehabilitation robot, this paper proposes a realization method of the adaptive force field of plane arbitrary training trajectory based on admittance control. In ROS, the flexible drag of the robot end is realized through the admittance control algorithm. For the drag trajectory, the function equation is obtained by fitting, and then the desired pose is adjusted in real time through the end pose to realize the trajectory force field. For the regular trajectory force field, a simpler plane segmentation method is used to achieve. The size of the force field is adaptively adjusted by means of position-force control, and the force is compensated in the tangential direction of the desired position, so as to realize the active rehabilitation training of the adaptive force field. The experimental results show that adding an adaptive force field to any plane training trajectory can improve training compliance, safety and training pertinence, which has an important reference for the application of personalized compliance active training to desktop upper limb rehabilitation robots.

Gao Lin, Dao-Hui Zhang, Ya-Lun Gu, Xin-Gang Zhao
FES-Based Hand Movement Control via Iterative Learning Control with Forgetting Factor

Functional electrical stimulation (FES) is an effective approach to restore hand movement function for patients with stroke. In this paper, a multi-electrode hand rehabilitation system is presented. Iterative learning control (ILC) with forgetting factor algorithm is employed to achieve an accuracy position control of multi-joint hand movement. A mapping matrix is identified to model the gains from the multi-electrode inputs to the multiple joints of the hand. The convergence conditions of ILC with forgetting factor for the proposed method are analyzed. Finally, experiments on healthy subjects are carried out to verify the performance of the proposed control method.

Guangyu Zhao, Qingshan Zeng, Benyan Huo, Xingang Zhao, Daohui Zhang

Actuation, Sensing and Control of Soft Robots

Frontmatter
3D Printing of PEDOT:PSS-PU-PAA Hydrogels with Excellent Mechanical and Electrical Performance for EMG Electrodes

Bioelectronics has been developed for recording the electrophysiological activity of diagnostic and therapeutic devices. However, current bioelectrodes still imperfectly comply with tissues, which results in high interfacial impedance and even mechanical detachment. Herein, we report a simple yet effective approach to overcome such hurdles by designing a highly conductive, adhesive hydrogel composite based on freeze-dried poly(3,4-ethylenedioxythiophene):poly (styrene sulfonate) (PEDOT:PSS), polyurethane (PU), and poly(acrylic acid) (PAA). With the continuous phase-separation of PEDOT:PSS, PU, and PAA, the resultant composite hydrogels can simultaneously achieve high adhesion (lap-shear strength > 8 kPa), stretchability (fracture strain > 1100%), and electrical conductivity (conductivity > 2 S/m) by overcoming the traditional trade-off between mechanical and electrical properties in conducting polymer hydrogels. Moreover, such hydrogels are readily applicable to advanced manufacturing techniques such as 3D printing. We further fabricated skin electrodes and achieved high quality and high signal-to-noise ratio EMG signal recording of the forearm.

Hude Ma, Jingdan Hou, Wenhui Xiong, Zhilin Zhang, Fucheng Wang, Jie Cao, Peng Jiang, Hanjun Yang, Ximei Liu, Jingkun Xu
Stretchable, Conducting and Large-Range Monitoring PEDOT: PSS-PVA Hydrogel Strain Sensor

Highly stretchable conducting polymer hydrogel strain sensors are widely used in many wearable electronic devices such as human exercise health monitoring, human-machine interface, and electronic skin. Flexible strain sensors can convert the sensed mechanical tensile deformation into electrical signal output. The structure and detection principle are simple. However, the strain sensors reported so far still face the problems of low stretchability, poor mechanical property and low sensitivity. In this work, we prepared an anisotropic tough conducting polymer hydrogel via a simple ice template-soaking strategy. The ice template can effectively control the growth of ice crystals, thereby forming a honeycomb-like micro-nano structure network; then the frozen hydrogel is immersed in a high concentration of sodium citrate salt solution. During the soaking process, phase separation of PEDOT:PSS and strong aggregation and crystallization of PVA were induced. The prepared conductive polymer hydrogels have excellent tensile properties, mechanical property and stable resistance changes. Conductive polymer hydrogels can be used as wearable strain sensors for detection of minute physiological movements and motion monitoring under large strains.

Zhilin Zhang, Hude Ma, Lina Wang, Xinyi Guo, Ruiqing Yang, Shuai Chen, Baoyang Lu
A Virtual Force Sensor for Robotic Manipulators Based on Dynamic Model

In human-robot interactions, a force sensor should be equipped in the robot to detect the force or torque to guarantee the safety of operators. However, mounting force sensors will increase the manufacturing cost. In this paper, a virtual force sensor that only requires motion information is designed to deal with this problem. Firstly, parameter identification is performed to obtain the dynamic model of the robot. After that, a virtual force sensor based on generalized momentum is designed to estimate the disturbance force or torque. Experiments are performed in a UR5 robot in the simulation software Coppeliasim, and the results validate that the virtual force sensor can estimate the force or torque in joint space.

Yanjiang Huang, Jianhong Ke, Xianmin Zhang
A Variable Stiffness Soft Actuator with a Center Skeleton and Pin-Socket Jamming Layers

Soft actuators have benefits of compliance and adaptability compared to their rigid counterparts. However, most soft actuators have finite enough stiffness to exert large forces on objects. This article proposes a tendon-driven soft actuator with a center skeleton and pin-socket jamming layers, allowing variable stiffness under different input air pressures. Two tendons drive the actuator to bend. The center skeleton reduces the required layers (1.6 mm total thickness) for a wide range of variable stiffness. The pin-socket design restricts the unexpected movement of layers and broadens its maximum angle when unjammed. Then, we develop a prototype and experimentally characterize its variable stiffness. Finally, we make a gripper with three proposed actuators to demonstrate the validity of the development.

Rong Bian, Ningbin Zhang, Xinyu Yang, Zijian Qin, Guoying Gu
A Parameter Optimization Method of 3D Printing Soft Materials for Soft Robots

With the increasingly complex structure design of soft robot, 3D printing of soft materials is one of the main research directions of soft robot manufacturing, which will give it new opportunities. Firstly, a soft material 3D printing platform compatible with embedded printing and external printing was built in this paper. Secondly, through the analysis and research of printing materials, pressure-flow rate simulation results of the different needles and the parameter experiment of printing influencing factors, the best combinations of printing parameters of functional magnetic material embedded printing and single component silica gel external printing were obtained respectively. Finally, based on this, bionic gecko magnetic drive soft robots and soft grippers were manufactured, and relevant performance experiments were carried out to verify the feasibility of integrated manufacturing of fully flexible soft robots using soft material 3D printing technology, highlighting the significant advantages of direct manufacturing and flexible model changing of 3D printing.

Anqi Guo, Wei Zhang, Yin Zhang, Lining Sun, Guoqing Jin
A Novel Soft Wrist Joint with Variable Stiffness

This paper presents a novel soft wrist joint made up of two soft variable stiffness bending joints and one soft torsion joint. The soft variable stiffness bending joint can achieve bending motion with variable stiffness and the motion of the soft bending actuator does not influence the terminal position of the soft bending joint, which provides a prerequisite for subsequent accuracy control. The torque of the soft torsion actuator is generated from four uniformly distributed spiral chambers and fiber-reinforced ropes, which can enhance torsion. The soft wrist joints are obtained by assembling them in series, providing a new solution for the soft robotic arm.

Gang Yang, Bing Li, Yang Zhang, Dayu Pan, Hailin Huang
Rangefinder-Based Obstacle Avoidance Algorithm for Human-Robot Co-carrying

The mobile robot (follower) in the human-robot co-carrying task needs to follow the human partner (leader) online, and also needs to avoid obstacles. Most existing local reactive obstacle avoidance algorithms cannot be applicable. This paper proposes a rangefinder-based obstacle avoidance algorithm for omnidirectional mobile robots in human-robot co-carrying. The proposed obstacle avoidance algorithm is implemented using a rangefinder, and obstacle avoidance can be achieved combined with controller design on our designed co-carrying system. Finally, simulation results are carried out to show the effectiveness of our proposed obstacle avoidance algorithm.

Xiong Guo, Xinbo Yu, Wei He
Design and Analysis of a Novel Magnetic Adhesion Robot with Passive Suspension

The steel lining of large facilities is an important structure that experiences extreme environments and requires periodical inspection after manufacture. However, due to the complexity of their internal environments (crisscross welds, curved surfaces, etc.), high demands are placed on stable adhesion and curvature adaptability. This paper presents a novel wheeled magnetic adhesion robot with passive suspension named NuBot, which is mainly applied in nuclear power containment. Based on the kinematic model of the 3-DOF independent suspension, a comprehensive optimization model is established, and global optimal dimensions are properly chosen from performance atlases. Then, a safety adhesion analysis considering non-slip and non-overturning condition is conducted to verify the magnetic force meet the safety demand. Experiments show that the robot can achieve precise locomotion on both strong and weak magnetic walls with different inclination angles, and can stably cross the 5 mm weld seam. Besides, its maximum payload capacity reaches 3.6 kg. Results show that NuBot has good comprehensive capabilities of surface-adaptability, adhesion stability, and payload. Besides, the robot can be applied in more ferromagnetic environments with more applications and the design method offers guidance for similar wheeled robots with passive suspension.

Hao Xu, Youcheng Han, Weizhong Guo, Mingda He, Yinghui Li

Machine Vision for Intelligent Robotics

Frontmatter
Pose Estimation of 3D Objects Based on Point Pair Feature and Weighted Voting

3D object pose estimation is an important part of machine vision and robot grasping technology. In order to further improve the accuracy of 3D pose estimation, we propose a novel method based on the point pair feature and weighted voting (WVPPF). Firstly, according to the angle characteristics of the point pair features, the corresponding weight is added to the vote of point pair matching results, and several initial poses are obtained. Then, we exploit initial pose verification to calculate the coincidence between the model and the scene point clouds after the initial pose transformation. Finally, the pose with the highest coincidence is selected as the result. The experiments show that WVPPF can estimate pose effectively for a 60%–70% occlusion rate, and the average accuracy is 17.84% higher than the point pair feature algorithm. At the same time, the WVPPF has good applicability in self-collected environments.

Sen Lin, Wentao Li, Yuning Wang
Weakly-Supervised Medical Image Segmentation Based on Multi-task Learning

Medical image segmentation plays an important role in the diagnosis and treatment of diseases. Using fully supervised deep learning methods for medical image segmentation requires large medical image datasets with pixel-level annotations, but medical image annotation is time-consuming and labor-intensive. On this basis, we study a medical image segmentation algorithm based on image-level annotation. Existing weakly supervised semantic segmentation algorithms based on image-level annotations rely on class activation maps (CAM) and saliency maps generated by preprocessing as pseudo-labels to supervise the network. However, the CAMs generated by many existing methods are often over-activated or under-activated, while most of the saliency map generated by preprocessing is also rough and cannot be updated online, which will affect the segmentation effect. In response to this situation, we study a weakly supervised medical image segmentation algorithm based on multi-task learning, which uses graph convolution to correct for improper activation, and uses similarity learning to update pseudo-labels online to improve segmentation performance. We conducted extensive experiments on ISIC2017 skin disease images to validate the proposed method and experiments show that our method achieves a Dice evaluation metric of 68.38% on this dataset, show that our approach outperforms state-of-the-art methods.

Xuanhua Xie, Huijie Fan, Zhencheng Yu, Haijun Bai, Yandong Tang
A Deep Multi-task Generative Adversarial Network for Face Completion

Face completion is a challenging task that requires a known mask as prior information to restore the missing content of a corrupted image. In contrast to well-studied face completion methods, we present a Deep Multi-task Generative Adversarial Network (DMGAN) for simultaneous missing region detection and completion in face imagery tasks. Specifically, our model first learns rich hierarchical representations, which are critical for missing region detection and completion, automatically. With these hierarchical representations, we then design two complementary sub-networks: (1) DetectionNet, which is built upon a fully convolutional neural net and detects the location and geometry information of the missing region in a coarse-to-fine manner, and (2) CompletionNet, which is designed with a skip connection architecture and predicts the missing region with multi-scale and multi-level features. Additionally, we train two context discriminators to ensure the consistency of the generated image. In contrast to existing models, our model can generate realistic face completion results without any prior information about the missing region, which allows our model to produce missing regions with arbitrary shapes and locations. Extensive quantitative and qualitative experiments on benchmark datasets demonstrate that the proposed model generates higher quality results compared to state-of-the-art methods.

Qiang Wang, Huijie Fan, Yandong Tang
Transformer Based Feature Pyramid Network for Transparent Objects Grasp

Transparent objects like glass bottles and plastic cups are common in daily life, while few works show good performance on grasping transparent objects due to their unique optic properties. Besides the difficulties of this task, there is no dataset for transparent objects grasp. To address this problem, we propose an efficient dataset construction pipeline to label grasp pose for transparent objects. With Blender physics engines, our pipeline could generate numerous photo-realistic images and label grasp poses in a short time. We also propose TTG-Net - a transformer-based feature pyramid network for generating planar grasp pose, which utilizes features pyramid network with residual module to extract features and use transformer encoder to refine features for better global information. TTG-Net is fully trained on the virtual dataset generated by our pipeline and it shows 80.4% validation accuracy on the virtual dataset. To prove the effectiveness of TTG-Net on real-world data, we also test TTG-Net with photos randomly captured in our lab. TTG-Net shows 73.4% accuracy on real-world benchmark which shows remarkable sim2real generalization. We also evaluate other main-stream methods on our dataset, TTG-Net shows better generalization ability.

Jiawei Zhang, Houde Liu, Chongkun Xia
Building Deeper with U-Attention Net for Underwater Image Enhancement

The images captured in the underwater scene suffer from color casts due to the scattering and absorption of light. These problems severe interfere many vision tasks in the underwater scene. In this paper, we propose a Deeper U-Attention Net for underwater image enhancement. Different from most existing underwater image enhancement methods, we adequately exploit underlying complementary information of different scales, which can enrich the feature representation from multiply perspectives. Specifically, we design a novel module with self-attention module to enhance the features by the features itself. Then, we design U-Attention block to extract the features at a certain level. At last, we build deeper two level U-structure net with the proposed U-attention block at multiply scale. This architecture enables us to build a very deep network, which can extract the multi-scale features for underwater image enhancement. Experimental results show our method has a better performance on public datasets than most state-of-the-art methods.

Chi Ma, Hui Hu, Yuenai Chen, Le Yang, Anxu Bu
A Self-attention Network for Face Detection Based on Unmanned Aerial Vehicles

Face detection based on Unmanned Aerial Vehicles (UAVs) faces following challenges: (1) scale variation. When the UAVs fly in the air, the size of faces is different owing to the distance, which increases the difficulty of face detection. (2) lack of specialized face detection datasets. It results in a sharp drop in the accuracy of algorithm. To address these two issues, we make full advantage of existing open benchmarks to train our model. However, the gap is too huge when we adapt face detectors from the ground to the air. Therefore, we propose a novel network called Face Self-attention Network (FSN) to achieve high performance. Our method conducts extensive experiments on the standard WIDER FACE benchmark. The experimental results demonstrate that FSN can detect multi-scale faces accurately.

Shunfu Hua, Huijie Fan, Naida Ding, Wei Li, Yandong Tang
Video Abnormal Behavior Detection Based on Human Skeletal Information and GRU

Video abnormal behavior detection is an important research direction in the field of computer vision and has been widely used in surveillance video. This paper proposes an abnormal behavior detection model based on human skeletal structure and recurrent neural network, which uses dynamic skeletal features for abnormal behavior detection. The model in this paper extracts key points from multiple frames through the human pose estimation algorithm, obtains human skeleton information, and inputs it into the established autoencoder recurrent neural network to detect abnormal human behavior from video sequences. Compared with traditional appearance-based models, our method has better anomaly detection performance under multi-scene and multi-view.

Yibo Li, Zixun Zhang
A Flexible Hand-Eye and Tool Offset Calibration Approach Using the Least Square Method

Hand-eye calibration is the basis of machine vision. The calibration method determines the motion accuracy of the manipulator. In the presented method, the point coordinate in camera frame and robot frame are separately obtained. By stacking the formula, the hand-eye transformation matrix can be calculated using the least square method. Otherwise, a tool offset calibration method is presented and some guidance also of conducting the tool offset calibration experiment is given. Finally, the validity of the proposed method is demonstrated by and experimental studies.

Jintao Chen, Benliang Zhu, Xianmin Zhang
Weakly Supervised Nucleus Segmentation Using Point Annotations via Edge Residue Assisted Network

Cervical cell nucleus segmentation can facilitate computer-assisted cancer diagnostics. Due to the obtaining manual annotations difficulty, weak supervision is more excellent strategy with only point annotations for this task than fully supervised one. We propose a novel weakly supervised learning model by sparse point annotations. The training phase has two major stages for the fully convolutional networks (FCN) training. In the first stage, coarse mask labels generation part obtains initial coarse nucleus regions using point annotations with a self-supervised learning manner. For refining the output nucleus masks, we retrain ERN with an additional constraint by our proposed edge residue map at the second stage. The two parts are trained jointly to improve the performance of the whole framework. As experimental results demonstrated, our model is able to resolve the confusion between foreground and background of cervical cell nucleus image with weakly-supervised point annotations. Moreover, our method can achieves competitive performance compared with fully supervised segmentation network based on pixel-wise annotations.

Wei Zhang, Xiai Chen, Shuangxi Du, Huijie Fan, Yandong Tang
An Analysis of Low-Rank Decomposition Selection for Deep Convolutional Neural Networks

Deep convolutional neural networks have achieved state of the art results in many image classification tasks. However, the large amount of parameters of the network limit its deployment to storage space limited situations. Low-rank decomposition methods are effective to compress the network, such as Canonical Polyadic decomposition and Tucker decomposition. However, most low-rank decomposition based approaches cannot achieve a satisfactory balance between the classification accuracy and compression ratio of the network. In this paper, we analyze the advantages and disadvantages of Canonical Polyadic and Tucker decomposition and give a selection guidance to take full advantage of both. And we recommend to use Tucker decomposition for shallow layers and Canonical Polyadic decomposition for deep layers of a deep convolutional neural network. The experiment results show that our approach achieves the best trade-off between accuracy and parameter compression ratio, which validates our point of view.

Baichen Liu, Huidi Jia, Zhi Han, Xi’ai Chen, Yandong Tang
Accurate Crop Positioning Based on Attitude Correction of Weeding Robot

In order to obtain accurate hoeing distance and reduce the influence of external factors such as vibration and robot platform tilt, this paper proposes a crop positioning point correction algorithm based on attitude information fusion. The algorithm uses the attitude information to calculate the mapping relationship between the image position under the tilt of the target crop camera and the actual ground position, which corrects the crop positioning point. The experimental results show that compared with the uncorrected positioning point, the average error of visual measurement of hoeing distance is reduced by about 58% and 73%, respectively. This algorithm can effectively improve the measurement accuracy of hoeing distance and reduce the possibility of seedling injury.

Tianjian Wang, Hui Zhou, Jiading Zhou, Pengbo Wang, Feng Huang
Low-Rank Tensor Tucker Decomposition for Hyperspectral Images Super-Resolution

Super-resolution is an important way to improve the spatial resolution of Hyperspectral images (HSIs). In this paper, we propose a super-resolution method based on low-rank tensor Tucker Decomposition and weighted 3D total variation (TV) for HSIs. Global tensor Tucker decomposition and weighted 3D TV regularization are combined to exploit the prior knowledge of data low-rank information and local smoothness. Meanwhile, we use log-sum norm for tensor Tucker Decomposition to approximate the low-rank tensor. Extensive experiments show that our method outperforms some state-of-the-art methods on public HSI dataset.

Huidi Jia, Siyu Guo, Zhenyu Li, Xi’ai Chen, Zhi Han, Yandong Tang
Hybrid Deep Convolutional Network for Face Alignment and Head Pose Estimation

Face alignment has been an important focus of vision research because it is the most fundamental step in face analysis, reconstruction, and applications of emotion and attention. However, face alignment still suffers from some problems, such as lack of stability and poor performance in practical applications due to occlusion, illumination, and high training costs. This paper proposes a Dual-Task Hybrid Deep Convolutional Network (DHDCN) to estimate head pose and facial landmark locations simultaneously. By connecting the multi-level features, the local features and global features can be effectively fused. Features common to both tasks are learned in the initial stages of the network, and later stages will train the two tasks independently. Although the results have some gaps compared to the state-of-the-art results, it also demonstrates the feasibility and potential of learning both tasks simultaneously.

Zhiyong Wang, Jingjing Liu, Honghai Liu
Research on Multi-model Fusion Algorithm for Image Dehazing Based on Attention Mechanism

In recent years, the researchers of image dehazing mainly focused on deep learning algorithms. However, due to the defective network structure, and inadequate feature extraction, the deep learning algorithm still has many problems to be solved. In this paper, we fuse the physical models including haze imaging model with absorption compensation, multiple scattering imaging model and multi-scale retinex imaging model with convolutional neural network to construct the image dehazing network. Multiple scattering haze imaging model is used to describe the haze imaging process in a more consistent way with the physical imaging mechanism. And the multi-scale retinex imaging model ensures the color fidelity. In the network structure, multi-scale feature extraction module can improve network performance in terms of feature reuse. In the attention feature extraction module, the back-propagating of the important front features is used to enhance features. This method can effectively make up for the deficiency that autocorrelation features cannot share the deep-level information, which is also effective for features replenishment. The results of the comparative experiment demonstrate that our network outperforms state-of-the-art dehazing methods.

Tong Cui, Meng Zhang, Silin Ge, Xuhao Chen
Study on FTO of Permanent Magnet Synchronous Motor for Electric Aircraft Steering Gear

As the core part of steering gear in electric aircraft, the failure of motor will threaten flight safety. Aiming at the fault problem of permanent magnet synchronous motor in the steering gear of electric aircraft, a fault-tolerant control (FTC) method based on three-phasefour-switch is proposed. The “Five-stage” SVPWM modulation method is designed to improve the problem of small flux freedom existing in the “Three-stage” SVPWM method, improve the degree of freedom and reduce the switching loss. Further combined with the sliding mode control idea, an improved sliding mode speed controller was designed to replace the traditional PI controller with fixed parameters. Thus, the improved sliding mode FTC strategy of electric aircraft permanent magnet synchronous motor based on “Five-stage” three-phasefour-switch was obtained to improve the robustness and rapidness of the system. The simulation results show that the proposed method can guarantee the stable operation of the steering gear and the flight safety of the electric aircraft even when the system inverter is faulty.

Meng Zhang, Tong Cui, Haoya Zhang, Nan Gao
A Classification Method for Acute Ischemic Stroke Patients and Healthy Controls Based on qEEG

Stroke has been one of the diseases with high incidence in the world, which is the main reason for adult deformity. It’s significant to diagnose stroke quickly and accurately. Currently the main diagnostic method of acute ischemic stroke is also Computed Tomography. Meanwhile, electroencephalogram (EEG) is an electrophysiological manifestation that directly reflects brain activity. Through the analysis of EEG, a large amount of physiological and pathological information can be found. Using qEEG as an indicator for the diagnosis of stroke patients has become a novel and prospective method. This paper proposed a classification method for stroke patients and healthy controls using Quadratic Discriminant Analysis. Four simple features of task-EEG which are RPR of Beta, Delta, DAR and DTABR were obtained by Welch’s method. The classification results showed the certain potential for qEEG as an diagnosis method.

Xiangyu Pan, Hui Chang, Honghai Liu

Micro or Nano Robotics and Its Application

Frontmatter
Non-destructive Two-Dimensional Motion Measurement of Cardiomyocytes Based on Hough Transform

Cardiomyocytes, as one of the few biological cells capable of autonomous beating in vitro, have attracted more and more attention in the fields of biology and robotics. In order to better understand the internal control mechanism of cardiomyocytes as a driver, scholars at home and abroad have proposed many measurement methods for the motion parameters of cardiomyocytes in the past 10 years, and the most common one is two-dimensional measurement. However, existing 2D cardiomyocyte measurement methods are always limited by substrate materials or cannot maintain long-term nontoxic measurements. Here, we proposed in-plane beating measurements of cardiomyocytes based on the microsphere Hough transform under a bright field. Through off-line processing, accurate tracking of the cardiomyocyte beating cycle can be achieved. Due to the simplicity, non-toxicity, and efficacy of the proposed protocol, this method will enable more rapid and accurate detection of cardiomyocyte beating. After further optimizing the data acquisition and data processing methods, we believe that this method will provide a more efficient and useful idea for real-time cardiomyocyte motion measurement.

Si Tang, Jialin Shi, Huiyao Shi, Kaixuan Wang, Chanmin Su, Lianqing Liu
Bubble Based Micromanipulators in Microfluidics Systems: A Mini-review

Bubbles in liquid have the advantages of controllability, compressibility and biocompatibility, so they are introduced into microfluidic system to drive the fluid and operate micro-objects including cells. In recent years, the acoustic and optothermal bubbles are the two most widely used and efficient bubbles in microfluidic devices. Therefore, the aim of this study is to review recent advances in acoustic bubble-based micromanipulators and optothermal bubble-based micromanipulators in microfluidic systems. The principles and applications of fluid control and micro-object operation of these two kinds of bubble-based manipulators are introduced and the prospects and challenges are discussed.

Yuting Zhou, Liguo Dai, Niandong Jiao, Lianqing Liu
Fast Locomotion of Microrobot Swarms with Ultrasonic Stimuli in Large Scale

In recent years, the acitve motion of micro-nano machines has received wide attention due to its potential applications in biomedicine, biosensing, pollution control and other fields. However, the rapid manipulation of microrobot swarms in a large range is still an unmet challenge. With merits of simple structure, rapid response, and biocompatibility, ultrasonic power has been emerging as one of the most suitable methods to accomplish locomotion for microrobots. Here we demonstrate a fast moving strategy of microrobot swarms with ultrasonic stimuli in a relatively large scale. Both aggregation and dispersion patterns are conducted in a manipulation reservoir with 6 mm in diameter. The aggregation of microrobot swarms is achieved by acoustic radiation force caused by acoustic pressure gradient, which drives the microrobots migrate towards the nearest sound pressure node. The dispersion behavior of the microrobots is driven by the acoustofluidics, which is caused by the localized traveling wave motion of the vibrating substrate under ultrasonic stimuli. Both modes of swarming are completed in several seconds, indicating the great promise for rapidly steering microrobot assembly in diverse potential applications.

Cong Zhao, Xiaolong Lu, Ying Wei, Huan Ou, Jinhui Bao
A Novel Acoustic Manipulation Chip with V-shaped Reflector for Effective Aggregation of Micro-objects

Micro-objects to aggregate at a specific location becomes a necessity when the concentration of micro-objects needs to be increased in the local area, such as in lab-on-a-chip devices. However, efficient aggregation is still a big challenge when detecting low concentrations of specimens on manipulation chips. Here, we present an acoustic manipulation chip containing artificial reflector to enhance the aggregation effect. To integrate with the traditional acoustic manipulation chips, V-shaped reflector is introduced into the microchannel as a solid vibration isolator to isolate acoustic waves in a liquid environment. When the V-shaped reflector is immersed into the sample contained in the microchannel to a certain depth, the activation of the piezoelectric transducer will produce acoustic oscillation of the substrate and localized acoustic streaming around the tip of the reflector. Under the localized acoustic streaming, micro-objects in the working area of the acoustic manipulation chip gather beneath reflector. Experimental results illustrate the proposed acoustic chip coupled with the V-shaped reflector offers an effective aggregation strategy for micro-objects, which has broad application prospects in the field of micro-manipulation.

Huan Ou, Xiaolong Lu, Ying Wei, Cong Zhao, Jinhui Bao

Biosignal Acquisition and Analysis

Frontmatter
An Adaptive Robust Student’s t-Based Kalman Filter Based on Multi-sensor Fusion

In practical applications, Kalman filter and its variants such as UKF may suffer from the time-varying measurement noise and process-error. Especially, when the process-error is heavy-tailed probability distribution, the Gaussian assumption would be no longer as accurately as expected. Aiming at the time-varying measurement noise and the situation with heavy-tailed process-error problems, in this paper, a new algorithm is proposed based on Student’s t-distribution and multi-sensor information fusion. The robustness of the proposed algorithm is guaranteed by the timely estimation of the measurement noise, and the adaptiveness is realized by replacing the Gaussian by the Student’s t-distribution. The Kullback-Leible Divergence (KLD) is used as the criterion for distinguishing the Gaussian distribution from the Student’s t-distribution. Finally, a challenging target tracking example is presented and the simulation results show that the proposed algorithm achieves a higher accuracy than the other algorithms.

Dapeng Wang, Hai Zhang, Hongliang Huang
Dynamic Hand Gesture Recognition for Numeral Handwritten via A-Mode Ultrasound

In recent years, due to the defects of weak sEMG signal, insensitive to fine finger movement and serious impression by noise, researchers consider the need to use A-mode ultrasound (AUS) for gesture decoding. However, the current A-mode ultrasonic gesture recognition algorithm is still relatively basic, which can recognize the recognition function of discrete gestures. However, due to the lack of time information, A-mode ultrasound still lacks an algorithm to recognize the dynamic gesture process. Therefore, we design and experiment a deep learning algorithm model applied to AUS signal, which is a deep learning framework based on LSTM. Due to the principle of LSTM, the model sets a certain number of frames as the whole action process, and constructs the connection of each frame in the whole process, so the time correlation (time characteristic) of AUS signal is constructed. Then, the features from AUS signal are sent to the complete full connection layer to output the classification results. And because AUS signal lacks data set of dynamic gestures, we designed and tested handwritten digits 0–9 as an example of dynamic gestures. Experimental results show that this algorithm can realize the dynamic gesture classification of AUS signal and solve the defect of AUS signal lacking time information. In addition, compared with the experimental action of traditional methods, it gives the practical significance of dynamic gesture in life, which is closer to life.

Donghan Liu, Dinghuang Zhang, Honghai Liu
Brain Network Connectivity Analysis of Different ADHD Groups Based on CNN-LSTM Classification Model

Attention deficit hyperactivity disorder (ADHD), as a common disease of adolescents, is characterized by the inability to concentrate and moderate impulsive behavior. Since the clinical level mostly depends on the doctor's psychological and environmental analysis of the patient, there is no objective classification standard. ADHD is closely related to the signal connection in the brain and the study of its brain connection mode is of great significance. In this study, the CNN-LSTM network model was applied to process open-source EEG data to achieve high-precision classification. The model was also used to visualize the features that contributed the most, and generate high-precision feature gradient data. The results showed that the traditional processing of original data was different from that of gradient data and the latter was more reliable. The strongest connections in both ADHD and ADD patients were short-range, whereas the healthy group had long-range connections between the occipital lobe and left anterior temporal regions. This study preliminarily achieved the research purpose of finding differences among three groups of people through the features of brain network connectivity.

Yuchao He, Cheng Wang, Xin Wang, Mingxing Zhu, Shixiong Chen, Guanglin Li
Heterogeneous sEMG Sensing for Stroke Motor Restoration Assessment

Stroke is a common global health-care problem that has had serious negative impact on the life quality of the patients. Motor impairment after stroke typically affects one side of the body. The main focus of stroke rehabilitation is the recovery of the affected neuromuscular functions and the achievement of independent body control. This paper proposes a heterogeneous sEMG sensing system for motor restoration assessment after stroke. The 136 sEMG nodes have been heterogeneously distributed to sense both dexterous and lump motion. The 128 sEMG nodes are clustered to acquire high density sEMG data acquisition; the other 8 are sparsely arranged for sEMG data acquisition in lump motion. Preliminary experiments demonstrate that the system outperforms existing similar systems and shows potential for evaluating motor restoration after stroke.

Hongyu Yang, Hui Chang, Jia Zeng, Ruikai Cao, Yifan Liu, Honghai Liu
Age-Related Differences in MVEP and SSMVEP-Based BCI Performance

With the aggravation of the aging society, the proportion of senior is gradually increasing. The brain structure size is changing with age. Thus, a certain of researchers focus on the differences in EEG responses or brain computer interface (BCI) performance among different age groups. Current study illustrated the differences in the transient response and steady state response to the motion checkerboard paradigm in younger group (age ranges from 22 to 30) and senior group (age ranges from 60 to 75) for the first time. Three algorithms were utilized to test the performance of the four-targets steady state motion visual evoked potential (SSMVEP) based BCI. Results showed that the SSMVEP could be clearly elicited in both groups. And two strong transient motion related components i.e., P1 and N2 were found in the temporal waveform. The latency of P1 in senior group was significant longer than that in younger group. And the amplitudes of P1 and N2 in senior group were significantly higher than that in younger group. For the performance of identifying SSMVEP, the accuracies in senior group were lower than that in younger group in all three data lengths. And extended canonical correlation analysis (extended CCA)-based method achieved the highest accuracy (86.39% ± 16.37% in senior subjects and 93.96% ± 5.68% in younger subjects) compared with CCA-based method and task-related component analysis-based method in both groups. These findings may be helpful for researchers designing algorithms to achieve high classification performance especially for senior subjects.

Xin Zhang, Yi Jiang, Wensheng Hou, Jiayuan He, Ning Jiang
HD-tDCS Applied on DLPFC Cortex for Sustained Attention Enhancement: A Preliminary EEG Study

This study investigated the effect of the high-definition transcranial direct current stimulation (HD-tDCS) on sustained attention reaction. The HD-tDCS was applied to the right dorsolateral prefrontal cortex (DLPFC) and the EEG signal was recorded simultaneously. We found that HD-tDCS could be used to enhance the reaction time and the N2 amplitude in response to the flanker stimuli changed significantly after the HD-tDCS stimulation. Moreover, the behavior enhancement was in accordance with the neural response. The HD-tDCS on DLPFC provides a way to enhance the subject’s attention to both the target task and the target task accompanied by distractors. The investigated approach may find application for digital medication in patients with attention problems, such as attention deficit hyperactivity disorder (ADHD).

Jiajing Zhao, Wenyu Li, Lin Yao
Reconstructing Specific Neural Components for SSVEP Identification

Brain-computer interfaces (BCIs) enable the brain to communicate directly with external devices. Steady-state visual evoked potential (SSVEP) is periodic response evoked by a continuous visual stimulus at a specific frequency in the occipital region of the brain. SSVEP-based BCI has been widely studied because of its high information transfer rate (ITR). However, SSVEP-BCI usually requires a long training period, which prevents its application. Therefore, it is necessary and meaningful to develop a decoding algorithm that can maintain high accuracy with less training time. This study proposed a point-position equivalent reconstruction (PPER) method to reconstruct stable periodic signals. The combination of PPER and ensemble task-related component analysis (eTRCA) algorithm can identify SSVEP with one training sample, whereas the classical methods need at least two training samples. Compared with eTRCA which achieved the highest average ITR of 172.93 ± 4.67 bits/min with two training samples, PPER-eTRCA can achieve the highest average ITR of 172.94 ± 3.75 bits/min with only one training sample. When the time window is 1 s, the accuracy of PPER-eTRCA with one training sample is significantly higher than the accuracy of eTRCA with two training samples (87.45% ± 1.19% vs. 85.32% ± 1.42%). The proposed method provides a feasible solution to reduce the training time while maintaining high performance, which will further facilitate the establishment of a more user-friendly SSVEP-BCI system.

Lijie Wang, Jinbiao Liu, Tao Tang, Linqing Feng, Yina Wei
Modeling and Recognition of Movement-Inducing Fatigue State Based on ECG Signal

Fatigue monitoring is significant during movement process to avoid body injury cased by excessive exercise. To address this issue, we developed an automated framework to recognize human fatigue states based on electrocardiogram (ECG) collected by a smart wearable device. After preprocessing on the raw ECG data, both machine learning solution and deep learning solution were introduced to recognize the fatigue states. Specifically, a set of hand-crafted features were designed which are fed into different machine learning models for comparison. For the deep learning solution, the residual mechanism was employed to build a deep neural network for fatigue classification. The proposed methods were evaluated on data collected from subjects after running exercise and achieved an accuracy of $$89.54\%$$ .

Jingjing Liu, Jia Zeng, Zhiyong Wang, Honghai Liu

Neurorobotics

Frontmatter
Structural Design and Control of a Multi-degree-of-freedom Modular Bionic Arm Prosthesis

Due to various reasons of natural disasters, car accidents, diseases and so on, different levels of amputations such as hand, wrist and shoulder disarticulation have been caused. A modular structural design of upper limb prosthesis that consists of hands, wrists, elbows, shoulders joints is vital to restore the lost motor functions of amputees and still remains a challenge. This paper designs a modular bionic arm prosthesis with five-degree-of-freedom according to the characteristics of weight, size and range of motion of a natural upper limb. By simulating and analyzing the kinematics of the arm prosthesis, results showed that the range of motion of the prosthesis is relatively wide and can meet the use of daily life. And based on the 3D printing technology, a whole arm prosthesis was printed and assembled modularly. Additionally, a control test of the modular arm prosthesis was conducted. The results showed that the designed prosthesis was operated successfully by the surface electromyography based pattern recognition control. The work of this study provides an effective modular bionic arm prosthesis structure that can restore different motor functions for patients with different levels of amputations.

Yingxiao Tan, Yue Zheng, Xiangxin Li, Guanglin Li
sEMG-Based Estimation of Human Arm Endpoint Stiffness Using Long Short-Term Memory Neural Networks and Autoencoders

Human upper limb impedance parameters are important in the smooth contact between stroke patients and the upper limb rehabilitation robot. Surface electromyography (sEMG) reflects the activation state of muscle and the movement intention of human body. It can be used to estimate the dynamic parameters of human body. In this study, we propose an estimation model combining long short-term memory (LSTM) neural network and autoencoders to estimate the endpoint stiffness of human arm from sEMG and elbow angle. The sEMG signal is a time varying nonlinear signal. Extracting key features is critical for fitting models. As an unsupervised neural network, autoencoders can select the proper features of sEMG for the estimation. LSTM neural network has good performance in dealing with time series problems. Through a 4-layer LSTM neural network, the mapping relationship between sEMG features and endpoint stiffness is constructed. To prove the superiority of the proposed model, the correlation coefficient between theoretical stiffness calculated by Cartesian impedance model and estimated stiffness and root mean square error (RMSE) is used as the evaluation standard. Compared with two other common models by experiments, the proposed model has better performance on root mean square error and correlation coefficient. The root mean square error and correlation coefficient of proposed model are 0.9621 and 1.732.

Yanan Ma, Quan Liu, Haojie Liu, Wei Meng
Error Related Potential Classification Using a 2-D Convolutional Neural Network

An error-related potentials (ErrP) is generated in the brain when human’s expectations are inconsistent with actual results. The decoding of ErrP can improve the performance of brain-computer systems (BCI). In this paper, we propose an effective ErrP classification method using the proposed attention-based convolutional neural network (AT-CNN). Every 1D EEG signal is transformed into a 2D grayscale image as an input data for the model. In addition, we introduced label smoothing to mitigate the impact of label mismatching data. We evaluate and compare our method using the Monitoring Error-Related Potential dataset. The accuracy of our proposed method is 83.42%, the sensitivity is 69.02%, the specificity is 88.48% and these results outperform the state-of-the-art methods.

Yuxiang Gao, Tangfei Tao, Yaguang Jia
A Multi-sensor Combined Tracking Method for Following Robots

At present, the research on tracking methods is mainly based on visual tracking algorithm, which has reduced the accuracy at night or under the condition of insufficient light intensity. Therefore, this paper starts from the direction of multi-sensor combined tracking. Firstly, in order to verify the feasibility and performance of the multi-sensor combined tracking method proposed in this paper, a set of tracking robot system is designed. Secondly, aiming at the problem that the visual tracking method fails to track in scenes such as complete occlusion and insufficient illumination, the non-line-of-sight perception of the following target is realized based on the fusion of ultra-wide band (UWB) and inertial measurement unit (IMU) sensors. Besides, based on coordinate transformation and decision tree algorithm, this paper makes decisions on UWB and visual tracking targets to achieve combined tracking.

Hao Liu, Gang Yu, Han Hu
Extracting Stable Control Information from EMG Signals to Drive a Musculoskeletal Model - A Preliminary Study

Musculoskeletal models (MMs) driven by electromyography (EMG) signals have been used to predict human movements. Muscle excitations of MMs are generally the amplitude of EMG, which shows large variability even when repeating the same task. The general structure of muscle synergies has been proved to be consistent across test sessions, providing a perspective for extracting stable control information for MMs. Although non-negative matrix factorization (NMF) is a common method for extracting synergies, the factorization result of NMF is not unique. In this study, we proposed an improved NMF algorithm for extracting stable control information of MMs to predict hand and wrist motions. Specifically, we supplemented the Hadamard product and L2-norm regularization term to the objective function of NMF. The proposed NMF was utilized to identify stable muscle synergies. Then, the time-varying profile of each synergy was fed into a subject-specific MM for estimating joint motions. The results demonstrated that the proposed scheme significantly outperformed a traditional MM and an MM combined with the classic NMF (NMF-MM), with averaged R and NRMSE equal to $$0.89\pm 0.06$$ and $$0.16\pm 0.04$$ . Further, the similarity between muscle synergies extracted from different training data revealed the proposed method’s effectiveness of identifying consistent control information for MMs. This study provides a novel model-based scheme for the estimation of continuous movements.

Jiamin Zhao, Yang Yu, Xinjun Sheng, Xiangyang Zhu
Construction of Complex Brain Network Based on EEG Signals and Evaluation of General Anesthesia Status

General anesthesia is now an important part of surgery, it can ensure that patients undergo surgery in a painless and unconscious state. The traditional anesthesia depth assessment mainly relies on the subjective judgment of the anesthesiologist, lacks a unified standard, and is prone to misjudgment. Since general anesthesia is essentially anesthesia of the central nervous system, the state of anesthesia can be monitored based on EEG analysis. Based on this, this paper proposes a method to reasonably construct a brain connection network system based on the characteristic parameters of EEG signals and combine machine learning to evaluate the state of anesthesia. This method extracts the EEG signals related to the depth of anesthesia, The knowledge of graph theory introduces the three functional indicators of Pearson correlation coefficient, phase-lock value and phase lag index to construct a complex brain network, and then perform feature selection based on the constructed brain network to generate a dataset, and use machine learning methods for classification. To evaluate the anesthesia state, the experimental results show that the accuracy of the method for evaluating the anesthesia state can reach 93.88%.

Zhiwen Xiao, Ziyan Xu, Li Ma
A Hybrid Asynchronous Brain-Computer Interface Combining SSVEP and EOG Signals for Rapid Target Recognition

Brain-Computer interfaces (BCIs) can help the disabled restore their ability of communicating and interacting with the environment. Asynchronous steady-steady visual evoked potential (SSVEP) allows the user to fully control the BCI system, but it also faces the challenge of the long erroneous state when the user switches his/her aimed target. To tackle this problem, a training-free SSVEP and EOG hybrid BCI system was proposed, where the spontaneous saccade eye movement occurred in SSVEP paradigm was hybridized into SSVEP detection by Bayesian approach to recognize the new target rapidly and accurately. The experiment showed that the proposed hybrid BCI had significantly higher asynchronous accuracy and short gaze shifting time compared to the conventional SSVEP-BCI. The results indicated the feasibility and efficacy of introducing the spontaneous eye movement information into SSVEP detection without increasing the user’s task burden.

Ximing Mai, Xinjun Sheng, Xiaokang Shu, Yidan Ding, Jianjun Meng, Xiangyang Zhu

Wearable Sensing and Robot Control

Frontmatter
ZNN-Based High-Order Model-Free Adaptative Iterative Learning Control of Ankle Rehabilitation Robot Driven by Pneumatic Artificial Muscles

As a new lightweight actuator, pneumatic artificial muscles (PAMs) have been widely used in rehabilitation robots attributing to the compliant interaction and good safety features. However, it is challenging to precisely control such PAMs-driven devices due to the non-linear and time-varying nature. For complex controlled plants like PAMs, the model-based controllers are challenging to design and use for their complex structure, while the convergence of the mode-free method can be further enhanced. In this paper, a Zero Neural Network based high-order model-free adaptive iterative control (ZNN-HOMFAILC) method is proposed to realize high precise position control of the rehabilitation robot. Firstly, the model of PAMs is converted to an equivalent linearized data model along the iterative axis. Then, to achieve fast convergence speed, a high-order pseudo partial derivative (PPD) law is designed to improve the convergence performance under different initial PPDs. The control law based on ZNN is designed to improve the learning ability from errors during iterations. Finally, the control performance and convergence speed of the ZNN-HOMFAILC are validated by simulation and actual control experiments of the PAMs-driven ankle rehabilitation robot. Results show that ZNN-HOMFAILC can significantly improve the convergence speed by 54% compared with MFAILC and HO-MFAILC, and the average tracking error of PAM can be reduced to a small level (2% of the range) after 7 iterations.

Xianliang Xie, Quan Liu, Wei Meng, Qingsong Ai
The Calibration of Pre-travel Error and Installation Eccentricity Error for On-Machine Probes

Touch-trigger probes are widely used in modern high-precision CNC machine tools as an on-machine inspection solution. The calibration of the probe pre-travel error is a significantly factor affecting the accuracy of the inspection system. However, the existing pre-travel calibration methods either have poor accuracy in practical application or require complicated extra equipments. In this paper, a simple but effective method for measuring the pre-travel error is proposed. With a high-precision standard ring gauge of known size, the center of the ring gauge is obtained by least square method fitting all the measured points. The eccentricity of the probe installation is separated by the reverse method. Then, the probe pre-travel table is established. In order to verify the accuracy of the pre-travel error table and the probe installation eccentricity, the spindle is rotated to touch the same point repeatedly on the high-precision plane. The experimental results show that the proposed novel calibration method can improve the accuracy considerably.

Jianyu Lin, Xu Zhang, Yijun Shen
A Preliminary Tactile Conduction Model Based on Neural Electrical Properties Analysis

The absence of tactile feedback leads to a high rejection rate from prostheses users and impedes the functional performance of dexterous hand prostheses. To effectively deliver tactile feedback, transcutaneous electrical nerve stimulation (TENS) has attracted extensive attention in the field of tactile sensation restoration, due to its advantages of non-invasive application and homology with neural signals. However, the modulation of electrotactile stimulation parameters still depends on operators’ experience instead of a theoretical guidance. Thus, this paper establishes a preliminary tactile conduction model which is expected to provide a theoretical foundation for the adjustment of electrotactile stimulation parameters. Based on a review of studies about the electrical conduction properties of electrodes and upper-limb tissues which are related to tactile generation process, a tactile conduction model is established to describe the neural signal transduction path from electrodes to tactile nerve fibres and the influence of different stimulation parameters on subjects’ sensation experience is briefly analysed.

Xiqing Li, Kairu Li
Backmatter
Metadaten
Titel
Intelligent Robotics and Applications
herausgegeben von
Honghai Liu
Zhouping Yin
Prof. Lianqing Liu
Li Jiang
Prof. Guoying Gu
Xinyu Wu
Weihong Ren
Copyright-Jahr
2022
Electronic ISBN
978-3-031-13822-5
Print ISBN
978-3-031-13821-8
DOI
https://doi.org/10.1007/978-3-031-13822-5

Premium Partner