Intelligent Robotics and Applications

Object-oriented navigation in unknown environments with only vision as input has been a challenging task for autonomous robots. Introducing semantic knowledge into the model has been proved to be an effective means to improve the suboptimal performance and the generalization of existing end-to-end learning methods. In this paper, we improve object-oriented navigation by proposing a knowledge-enhanced scene context embedding method, which consists of a reasonable knowledge graph and a designed novel 6-D context vector. The developed knowledge graph (named MattKG) is derived from large-scale real-world scenes and contains object-level relationships that are expected to assist robots to understand the environment. The designed novel 6-D context vector replaces traditional pixel-level raw features by embedding observations as scene context. The experimental results on the public dataset AI2-THOR indicate that our method improves both the navigation success rate and efficiency compared with other state-of-the-art models. We also deploy the proposed method on a physical robot and apply it to the real-world environment.

Recently, object detection technologies applied in rehabilitation systems are mainly based on the ready-made technology of CNNs. This paper proposes an DETR-based detector which is an end-to-end object detector with spatiotemporal context learning for machine-assisted rehabilitation. To improve the performance of small object detection, first, the multi-level features of the RepVGG are fused with the SE attention mechanism to build a SEFP-RepVGG. To make the encoder-decoder structure more suitable, next, the value of the encoder is generated by using feature maps with more detailed information than key/query. To reduce computation, Patch Merging is finally imported to modify the feature map scale of the input encoder. The proposed detector has higher real-time performance than DETR and obtains the competitive detection accuracy on the ImageNet VID benchmark. Some typical samples from the NTU RGB-D 60 dataset are selected to build a new limb-detection dataset for further evaluation. The results show the effectiveness of the proposed detector in the rehabilitation scenarios.

Skeleton-based hand gesture recognition has achieved great success in recent years. However, most of the existing methods cannot extract spatiotemporal features well due to the skeleton noise. In real applications, some large models also suffer from a huge number of parameters and low execution speed. This paper presents a lightweight skeleton-based hand gesture recognition network by using multi-input fusion to address those issues. We convey two joint-oriented features: Center Joint Distances (CJD) feature and Center Joint Angles (CJA) feature as the static branch. Besides, the motion branch consists of Global Linear Velocities (GLV) feature and Local Angular Velocities (LAV) feature. Fusing static and motion branches, a robust input can be generated and fed into a lightweight CNN-based network to recognize hand gestures. Our method achieves 95.8% and 92.5% hand gesture recognition accuracy with only 2.24M parameters on the 14 gestures and 28 gestures of the SHREC’17 dataset. Experimental results show that the proposed method outperforms state-of-the-art (SOAT) methods.

For use in unstructured domains, highly redundant robotic systems need both deliberative and compliant control schemes, to avoid collision and safely interact with the dynamic environment. Aiming at the shortcoming of the traditional method of path planning using merely on the typical structure of the manipulator, a new algorithm, named the “skeleton extraction based on 3D-depth camera”, is proposed for the real-time generation of collision avoidance motions. The algorithm is applied to get the distances of the multiple possible collision points and to establish a new form of a repulsive force, which includes the radial repulsive force and tangential repulsive force. For the redundant manipulator, the equilibrium angles through incremental iteration of the moment instead of inverse kinematics to reduce calculation cost. Finally, the method was tested by a 7-DOF manipulator in MATLAB environment. The results show that the proposed method can avoid local minima traps and eliminate oscillations effectively.

The Grasping of unknown objects is a challenging but critical problem in the field of robotic research. However, existing studies only focus on the shape of objects and ignore the impact of the differences in robot systems which has a vital influence on the completion of grasping tasks. In this work, we present a novel grasping approach with a dynamic annotation mechanism to address the problem, which includes a grasping dataset and a grasping detection network. The dataset provides two annotations named basic and decent annotation respectively, and the former can be transformed to the latter according to mechanical parameters of antipodal grippers and absolute positioning accuracies of robots. So that we take the characters of the robot system into account. Meanwhile, a new evaluation metric is presented to provide reliable assessments for the predicted grasps. The proposed grasping detection network is a fully convolutional network that can generate robust grasps for robots. In addition, evaluations based on datasets and experiments on a real robot show the effectiveness of our approach.

The quantity of tomatoes is closely related to their yield information, and a powerful inspection robot that can automatically count tomatoes is urgently necessary for the hot and harsh environment of greenhouses. With the continuous progress of computer vision technology, the use of deep learning algorithms for counting tomatoes can greatly improve the inspection speed of the inspection robot. This paper propose a tomato fruit counting method for greenhouse inspection robots, which tracks the position of tomatoes in the image by the spatial displacement information of the robot, while 3D depth filtering can effectively avoid the interference of background tomatoes on the counting. The main advantages of this method are: (1) it can realize the tracking of bunched fruits and the counting of single fruits at the same time; (2) it can avoid the interference of background tomatoes. The experimental results of the greenhouse showed that the accuracy rates of bunch and single fruit counting were higher than 84% and 86% respectively, which greatly improved the inspection speed compared with manual counting and basically meet the counting requirements of the current greenhouse.

Stand-to-sit movement(SST) refers to the process from stand to sit, and the interaction of SST usually refers to the process of contact between people and the seat. SST is a relaxed and natural movement for normal people with healthy lower limbs, and they will not perceive any difficulties. However, for those people with lower limb disabilities, they have certain handicaps in SST. When they perform SST, owing to their lower extremity muscles could not work properly, they would receive a violent shock from the seat after touching the seat surface, which is a huge threat to the safety of the interactor. This paper mainly studies the safety assistance method of the nursing robot in the process of contact between the human body and the seat in SSI. We propose a lumped parameter model of human-robot interaction based on the vertical dimension to describe the post-contact process. On the basis of completing the modeling, we constructed the safety auxiliary control system of the nursing robot, and verified the effectiveness of the control system through the simulation experiment of the control system.

Aiming at the problems of collision, non-smoothness, and discontinuity of the trajectory generated by the motion planning of the service robot in the home environment, this paper proposes a robot motion planning system consisting of three parts: path planning, trajectory generation, and trajectory optimization. First, use the A* algorithm based on graph search to quickly plan a passable global path in a complex home environment as the initial value of trajectory generation; secondly, construct an objective function based on Minimum Snap to generate the initial trajectory to be optimized; finally, Using the convex hull property of the Bezier curve, the safety corridor is constructed and the time distribution is adjusted by the trapezoidal velocity curve method, which solves the overshoot phenomenon that occurs in the solution process of the trajectory generation. The Minimum Snap method based on the Bezier Curve is constructed to optimize the trajectory and finally generate A continuous and smooth motion trajectory with minimal energy loss suitable for service robots. The feasibility and effectiveness of this method are proved by simulation experiments.

Identifying the wrong location is one of the prerequisites for mobile service robots to work safely and stably. This paper proposes a current location confidence algorithm (CLCA) suitable for mobile service robots in the elder-care environment. In this method, the laser data is matched with the grid map information to obtain the current location confidence, which is used in the safety module to identify the wrong location. Compared with other laser-based methods, the CLCA is not limited by the geometric features of the map, so it is particularly applicable for the elder-care environment with complex space scenes. To verify the feasibility of the CLCA, we have experimented with calculating the current location confidence of the robot. The experimental results show that this method can identify the wrong location and take safety measures.

To tackle behavioral assistance for elderly care in daily life, our laboratory rollout seven homecare robots: walking support robot, gait rehabilitation robot, intelligent wheelchair robot, excretory support robot, personal care robot, intelligent bed, and transport robot. By integrating the multi-heterogeneous robot systems (MHRS) and distributed sensor networks, we propose a novel smart home for elderly care that can cover most life behaviors. Furthermore, to realize that the MHRS can efficiently serve multi-user within the architecture of the smart home. A self-organizing MHRS architecture is proposed. This architecture combines specific tasks to establish a robot group communication mechanism, and multi-robot in the group can complete adaptive control according to the user’s real-time position, ensuring assist safety and adaptability. Finally, we conducted experiments in the proposed smart home to rising transfer, standing transfer, and behavior assistance. The experiments show the proposed smart home has the auxiliary capabilities for basic activities of daily living (ADLs), instrumental activities of daily living (IADLs), enhanced activities of daily living (EADLs) even continuous assistance in safety and comfortable way, which can be used in homes, hospitals, rehabilitation center and other scenes for elderly care.

The main idea of this paper is to design an attention quality testing system based on the combination of virtual reality technology and eye-tracking technology, through which a visual attention stability testing method is established and the feasibility of the system and method is verified. The relationship between attention and eye movements and prefrontal cerebral blood flow was investigated by simultaneous acquisition of prefrontal blood oxygen concentration in the attentional stability test experiment. After the experimental data analysis, the eye-movement signal can accurately capture the information of attention shifting occurred, and find the evidence that attention shifting excited the prefrontal brain area, and find the relationship between attention, eye-movement and prefrontal. It verified the rationality of the combined method of virtual reality technology and eye-tracking technology in the content of attentional stability test, and provided scientific data support and experimental suggestions for future attention deficit disorder to conduct attentional stability test.

Exploring the most economic method to achieve micro-low gravity simulation training has been troubling researchers for a long time. Hence, this paper presents an active lower limb exoskeleton to counteract gravity of astronauts with a very low cost compared to conventional ways. It includes two link rods, fixing modules, servo motors used in hip and ankle, which is designed as common structures of lower limb exoskeletons. Note that selected servo motors can feed back driving torque and rotating angle at the same time, which makes exoskeleton more integrated and light. Different from common control methods applied in assistive exoskeleton to help wearers walk or run, zero-force control is used in the exoskeleton to counteract gravity for micro-low gravity simulation training of astronauts. Healthy volunteers are recruited to wear this exoskeleton and their surface electromyography (sEMG) signals are recorded during the process. Experimental results show that the proposed lower limb exoskeleton can averagely counteract gravity of more than 60% for each subject in static and dynamic states.

Omnidirectional mobile robot is applied in people's daily life. In order to properly assist users, the robot must accurately track the predetermined trajectory. However, the robot's tracking accuracy is severely compromised center of gravity shifts induced by the user. An acceleration proportional differential control strategy based on parameter estimation has been proposed in this paper when the center of gravity of the robot is different from its geometric center. The present paper first investigates the dynamic of mechanical structure and constructs a new dynamic model by considering the interference of the center of gravity shift. Secondly, a parameter estimation strategy is designed to estimate the dynamic center of gravity in real time. Then, an acceleration proportional differential controller with center of gravity offset compensation is designed to control the robot. Next, based on Lyapunov stability theory, stability analysis is carried out to prove the asymptotic stability of the proposed control algorithm. Finally, simulation validation shows that the control accuracy of the proposed method is more accurate than proportional differential and adaptive control because they can estimate the center of gravity offset parameters in real time.

In order to realize the precise control of the contact force and position between the two manipulators and the object when the dual-arm robot cooperatively carry the target object, a modified sliding mode control method with force coordination performance based on impedance control is proposed in this paper. Firstly, the motion constraint relationship and dynamic model of the dual-arm robot are obtained according to the relative position relationship of the dual-arm robot when carrying objects. Next, the force of the controlled object is analyzed and decomposed by Newton’s second law, and the terminal contact force of dual arms is obtained. Then, we integrate the hyperbolic tangent sliding mode control algorithm into the position-based impedance control method to realize the dual control of force and position, and the stability and convergence analysis of the proposed cooperative control system is given. Finally, we verified through simulation that this control scheme can realize the precise control of the position and contact force the manipulator.

In order to relieve the medical pressure, when patients with finger disability see a doctor, the degree of finger disability can be identified and judged by the equipment first, and then the doctor carries out the next step of diagnosis and treatment. Aiming at the problem that the traditional recognition algorithm is not ideal, this paper proposes a finger disability recognition algorithm based on Holistically-nested edge detection algorithm. On the basis of extracting the edge of hand image with Holistically-nested edge detection algorithm, the similarity judgment is made between the experimental object’s hand edge detection image and the standard hand edge detection image. The degree of finger joint integrity was analyzed by different similarity judgment, and then the degree of finger disability was judged. In order to verify the effectiveness of the method, 50 people’s hand images were collected to establish a sample database of hand images, and a total of 600 simulated severed finger images were tested. The accuracy of finger disability recognition was 96.6%. This algorithm can effectively identify the degree of finger disability and improve the medical efficiency.

Emotion recognition is one of the key technologies for the further development of human-computer interaction, and is gradually becoming a hot spot in current AI research. At the same time, physiological signals are objective external manifestations of emotions, and emotion recognition based on physiological signals often lacks high-quality training samples and suffers from inter-sample category imbalance. In this paper, 140 samples of electrocardiogram (ECG) signals triggered by Self-Assessment Manikin (SAM) emotion self-assessment experiments were collected using International Affective Picture System (IAPS). To cope with the problem of small data size and data imbalance between classes, Wasserstein Generative Adversarial Network-Gradient Penalty (WGAN-GP) was used to add different number samples for different classes to achieve class balance in the training set, and by continuously adding -samples, the training set to different sizes, using three classifiers to train different sizes of training set samples separately. The results show that the accuracy and weighted F1 values of all three classifiers improve after increasing the data, where higher accuracy and F1 values can be obtained using the proposed Multi Attention-CNN(MA-CNN) as a classifier before and after increasing the samples.

For the optimization of an intelligent wheelchair driving path, most studies focus on the length of the path. Although a few studies have taken user preferences and comfort into account, the length of the path has not been involved in the meantime. Namely, we can say user preferences are not considered when considering path length, while no consideration of user preferences when considering path length, which leads to conflicts between the path length of intelligent wheelchairs and user preferences. Therefore, this study proposes a multi-objective optimization method for intelligent wheelchair path considering user preferences. Firstly, an intelligent wheelchair path preference recognition framework based on evidence network is proposed. Secondly, A* and artificial potential field fusion path planning method is used to generate a certain scale of paths, and the user's path preference and path length are calculated, so as to build a mathematical model for multi-objective optimization. Finally, the model is solved by multi-objective PSO. The results show that the proposed method can realize the optimization of the intelligent wheelchair path with consideration of user preference and path length.

With the development of deep convolutional neural networks, 2D CNN is widely used in action detection task. Although 2D CNN extracts rich features from video frames, these features also contain redundant information. In response to this problem, we propose Residual Channel-Spatial Attention module (RCSA) to guide the network what (object patterns) and where (spatially) need to be focused. Meanwhile, in order to effectively utilize the rich spatial and semantic features extracted by different layers of deep networks, we combine RCSA and deep aggregation network to propose Deep Attention Aggregation Network. Experiment resultes on two datasets J-HMDB and UCF-101 show that the proposed network achieves state-of-the-art performances on action detection.

In order to enable the unmanned motion platform to obtain real-time environmental semantic information and obstacle depth information, a real-time semantic segmentation and feature point matching based on binocular cameras are considered. This method firstly takes advantages of a real-time semantic segmentation network to obtain the road scene information and the region of obstacles on the road such as vehicles or pedestrians. Then, feature matching is performed on the region of interest (ROI) of left and right views. In the experiment part, firstly we conduct simulation verification on the KITTI dataset, and then we conduct binocular camera calibration, rectification, segmentation and stereo matching based on Oriented FAST and Rotated BRIEF (ORB) method on the actual system. The experiment results proves that the method is real-time and robust.

Magnetically controlled microrobots have attracted wide attention in noninvasive therapy. However, it is challenging to design a microrobot with both low motion resistance and multi-mode motions control. Here, we design a 100 μm helical drill-like microrobot with biodegradable materials GelMA and HAMA. The microrobot is optimized with surface pores to reduce the resistance and alternately rotates and oscillates in composite magnetic fields. Inspired by the dimpled surface of the golf ball to reduce the pressure drag via fluid transition, the microdrill is modified with 98 dimples over its surface to effectively reduce the movement resistance. Considering hyperviscosity tasks, a control strategy to dynamically switch rotating and oscillating composite magnetic fields is performed with visual recognition of the local environment, which actuates the microdrill to move flexibly. The experiment demonstrates that the swimming step-out frequency of the dimpled microdrill is improved 44.5% to 13 Hz, and swimming velocity of the dimpled microdrill is improved by 13.7% to 25.3 μm/s. Furthermore, the microdrills can be degraded by collagenase in a concentration of 0.35 mg/mL, which shows good biocompatibility and is anticipated to be applied in microsurgery and untethered therapies in the future. (This work was supported by National Key R&D Program of China under grant number 2019YFB1309701, and0 National Natural Science Foundation of China under grant number 62073042).

This paper designs an automatic calibration method and system of motor torque for the problem of low loading accuracy of motor torque. The system uses genetic algorithm to optimize PID parameters and load control and measurement of the motor. The genetic algorithm is realized in the simulation platform, and the iterative operation is carried out by setting different cross probability and mutation probability parameters. The results are substituted into the motor model to analyze the response speed and anti-interference ability of the motor to the given random signal, and the optimal PID parameters are obtained as the configuration parameters of the motor torque automatic calibration system. The experimental results show that compared with the traditional motor torque calibration loading control, the accuracy of the system torque calibration error is improved and the error range is controlled within ±0.003 N $$\cdot $$ · m, which verifies the effectiveness and feasibility of this method.

In order to improve the accuracy of the optimal solution obtained by the scheduling model in uncertain workshop environment, a multi-AGV scheduling model with obstacle impact factor is proposed. The multi-AGV scheduling model takes minimum total length of driving path of AGVs and maximum utilization rate of AGVs as the optimization goal and the obstacle impact factor is obtained according to the area and shape of obstacles. The genetic algorithm (GA) with a custom coding method is adopted to obtain the optimal solution of the multi-AGV scheduling model. The improved A* algorithm improves the quality of the driving path of AGVs transporting materials by setting virtual obstacle area and turning penalty factor. The simulation result shows the model that introduces obstacle impact factor is 41% more accurate than the model without obstacle impact factor.

A type synthesis method of six degrees of freedom (DOFs) parallel mechanism (PM) with decoupled translation and rotation (DTR) is proposed by analyzing the input and output characteristics of partially decoupled parallel mechanism (PDPM). Firstly, based on the requirements of Jacobian matrix of PDPM, the direct Jacobian matrix and inverse Jacobian matrix are constructed by the screw theory, so as to determine the actuation wrench screw (AWS) that represents the force or couple acting on the moving platform by the actuated twist screw (ATS) of the limb. According to the AWS and the connectivity, the ATS representing the driving pair and the non-actuated twist screw (NATS) representing the non-driving pair on the corresponding limb are obtained, and then the configuration of the limb structure screw system is completed. Finally, according to the limb combination principle of PDPM, six limbs are selected in turn to connect the moving platform and the fixed platform, and then a variety of six DOFs PMs with DTR are obtained. The six DOFs PM with DTR can be used as the main structure of the joint rehabilitation robot, which provides an idea to solve the problem that the rotation center of the robot joint is inconsistent with the actual physiological center of human joint. This kind of mechanism has the characteristics of compact structure and simple control that shows wide application prospects.

Aiming at the situation of external disturbance and actuator failure in the attitude control of quadrotor UAV, an adaptive fault-tolerant control (FTC) method based on finite-time disturbance observer is proposed. First, the UAV dynamic model is decoupled into attitude subsystem and position subsystem; a finite-time disturbance observer is designed to observe the external unknown disturbances and actuator fault signals in the system in real time, and the observations are combined with the design of a non-singular fast terminal sliding mode controller, which not only realizes the detection of the unknown external disturbances in the system. It suppresses and compensates for the influence of actuator failure, and improves the tracking speed and control accuracy of the system. The stability of the control system is proved based on Lyapunov theory. Finally, the effectiveness of the proposed method is verified by simulation.

In order to improve the dynamic motion ability of the biped robot, a joint with high torque output and high backdrivability is required. In this paper, a new leg mechanism using a joint with continuously variable reduction ratio inspired by human joint structure is proposed. This mechanism possesses high actuation capability and high impact resistance ability. Based on the characteristics of jumping motion, the parameters of the joint are optimized to increase the jumping height of the robot. A contrast simulation was implemented on a one-legged model to show the advantages of the variable reduction ratio joint over fixed reduction ratio joint. The newly designed joint can increase the jumping height of the robot by 21% comparing with a model without the mechanism. A prototype of one-legged robot using the designed joint with continuously variable reduction ratio has been developed. Vertical jump experiment on the prototype is realized with a height of 42 cm.

In order to realize the automatic measurement and positioning drilling of the end face’s hole of the turbine blade, a system of automatic measurement, positioning and drilling is built by using a binocular vision laser scanner, drilling tool, and industrial robot et al., and an adaptive method of position-pose for the robot to locate the end face’s hole of the blade is proposed. In this method, the laser scanning is used to measure the point cloud of the end face of the blade, and the position coordinates and the normal vector of the hole are determined. The drill calibration algorithm and hand-eye calibration algorithm are used to realize the parameterization of the system. Based on the coordinate system transformation principle, through the projection transformation of the normal vector determines the optimal pose of positioning the end face’s hole for the coordinate system of the drill tool central point, and the optimal position-pose solution of the robot carries the relevant equipment to locate the end face’s hole of any complex blade is automatically solved. The experimental results showed that the method of controlling the tool central point of the drill to locate the given position, the error was less than 1 mm, which was better than the 1.5 mm of the traditional method. The linear direction of the drill was highly perpendicular to the plane of the target drilling position, which met the operational requirement of the workpiece.

Aiming at the problem of low global positioning precision and a large number of reflectors in the global feature map, an AGV positioning algorithm for reducing the number of reflectors is proposed. First, the global feature map is constructed by the reflectors. Next, the reflection points are obtained by lidar scanning, in which abnormal reflection points are removed through preprocessing. The local coordinates of the reflector are clustered and fitted by combining the reflection intensity of the reflector point. Then, the global coordinates of the reflectors are obtained by matching the local coordinates of the reflector with the global feature map. Finally, the initial position of the AGV is obtained through the static pose calculation algorithm, and the dynamic position of the AGV is solved by the two-point positioning algorithm. The experimental results show that, compared with the traditional algorithm, the positioning algorithm based on reflectors in this paper decreases the global position precision by 42.0% and 16.1% in the X-axis and Y-axis, respectively, and the number of reflectors used for the positioning algorithm is reduced from three or more to two.

In this study, a local optimization method of the five-axis CNC machining tool axis vector based on kinematic constraints is proposed, which realizes the smoothing of the trajectory of the driving rotation axis in the machine tool coordinate system, so as to realize the smooth machining of the machine tool and reduce the occurrence of vibration. Firstly, this study proposes an optimization interval selection method based on kinematic parameters, that is, the tool path that does not meet the characteristics of the speed, acceleration or jerk of the rotating axis is defined as the tool path that needs to be optimized, and an algorithm based on bidirectional scanning is proposed to determine the start and end positions of each optimization interval. Secondly, the tool axis vector optimization method based on ruled surface is used to optimize the tool axis vector, and a ruled surface space is established at each tool position point, and the tool axis optimization is limited within a certain range, to minimize the dynamic characteristics of the rotary axis as the optimization goal to realize the optimization of the tool axis. Finally, the proposed method is verified by experiments, and the smoothing of the rotating drive shaft of the machine tool is realized.

Blisks with integral structure are key component of the power system. Blade profile usually possesses high complexity and is hard to be milled. There are problems of vague data system and low interconnection in the process information management, which seriously impedes the improvement of the intelligence of milling process. This paper presents a modular information schema in the view of machining features, and the data integration model of blade milling process information is established. Furthermore, the composition of blade milling process database is proposed. Take the advantage of the CAM software specially for 5-axis milling blade and relational database technology, a prototype software system for data management of blade milling process, which is able to realize the function of automatic extraction, storage, and correlation interaction for relevant process data, is developed. It has been tested and verified that the data management model and prototype system are feasible and available by collecting and storing the process data of some simulated blade profiles. The work in this paper can be accepted as a technical support to the integrating information management, and the machining efficiency improvement of intelligent milling unit for the complex blade parts.

In order to improve the obstacle-crossing ability, motion stability and load-bearing capacity of mobile robots for different terrains, the Rubik's Cube mechanism (RCM) with strong coupling and variable topology is introduced into the field of mobile robots, and a wheel-legged mobile robot (WLMR) based on RCM is proposed. A new type of chute third-order RCM is proposed and applied to the wheel-leg conversion module, then a WLMR with polymorphism is constructed by combining wheel-leg conversion module, mechanical leg and Mecanum wheel. Moreover, in order to ensure the stability of the robot during movement, gait planning analysis of the WLMR in different modes is carried out. Eventually, the prototype experiments are performed to verify the efficiency of the WLMR's straight travel, in-situ rotation, obstacle-crossing and morphology transformation in complex environments. This research not only provides a reference for the design of polymorphous mobile robots, but also opens up ideas for the application of the RCM in daily production and life.

With the exploration of the ocean, amphibious robots can integrate the advantages of underwater and land robots, and can achieve detection on land, underwater, and seabed. This topic proposed the idea of bionic waterfowl, designed a set of amphibious bionic waterfowl robot prototype, and built the machinery platform and control system platform. The robot’s dynamic leg and head and neck are moved by modeling; two kinds of land gait designs and simulation analysis of the robot are carried out by ADAMS software. In the simulation process, by adjusting the leg bending angle and joint rotation frequency of the two basic gaits designed, the robot can have a certain ability to overcome obstacles, and can run smoothly on horizontal ground and slopes with different angles. progress. Finally, according to the experimental results, the relationship curves between the leg bending angle and the anterior distance and the joint rotation frequency and the anterior distance were fitted. The bionic duck robot can choose the most suitable gait through the expression of fitting curve under different land environment conditions.

IS (Interpersonal Synchrony), where the follower (participant) tries to behave the same action along with the raiser (human or metronome), is an essential social interaction skill. The evaluation of interpersonal synchronization is valuable for early autism screening. However, the research on IS evaluation is limited, and the current approaches usually evaluate the IS task with “motion energy” that is calculated by imprecise corner detection of the participant, which is not robust in an uncontrollable clinical environment. Moreover, these approaches need to manually mark the start and the end anchor of the specified action segment, which is labor-intensive. In this paper, we construct a realtime action segmentation model to automatically recognize the human-wise action class frame by frame. A simple yet efficient backbone is utilized to classify action class straightly instead of extracting the motion features (e.g. optical flow) with high computational complexity. Specifically, given an action video, a sliding window stacks frames in a fixed window size to feed a Resnet-like action classification branch (ACB) to classify the current action label. To further improve the accuracy of action boundary and eliminate the over-segmentation noises, we incorporate a boundary prediction branch (BPB), cooperating with majority-voting strategy, to refine the action classification generated by ACB. Then we can calculate the IS overlap easily by comparing two action timelines belonging to raiser and follower. To evaluate the proposed model, we collect 200K annotated images belonging to 40 subjects who perform 2 tasks (nod and clap) in 2 conditions (interpersonal and human-metronome). The experiment results demonstrate that our model achieves 87.1% accuracy at 200 FPS and can locate the start and end of action precisely in realtime.

Nowadays, Autism spectrum disorder (ASD) is a neurodevelopmental disorder that severely affects social communication. The diagnostic criteria depend on clinicians’ subjective judgment of the patient’s behavioral criteria. Obviously, it is an urgent problem to establish an objective diagnosis method for patients with ASD. To address this problem, we propose a novel graph convolutional network(GCN) method based on relational attention mechanism. Firstly, we extract functional connectivity (FC) between brain regions from functional magnetic resonance (fMRI) effects that respond to blood oxygenation signals in the brain. Considering the different relationships between subjects, population relations are then modeled by graph structural models as a way to jointly learn population information. Finally, for individual-specific information, a relational attention mechanism is used to generate relationships between subjects and GCN is utilized to learn their unique representational information. Our proposed method is evaluated 871 subjects (including 403 ASD subjects and 468 typical control (TC) subjects) from the Autism Brain Imaging Data Exchange (ABIDE). The experimental results show that the mean accuracy and AUC values of our proposed method can obtained 90.57% and 90.51%, respectively. Our proposed method has achieved state-of-the-art performance in the diagnosis of ASD compared to some methods published in recent years. Overall, our method is effective and informative in guiding clinical practices.

In recent years, gaze estimation has been applied to numerous application areas, such as driver monitor system, autism assessment, and so on. However, current practical gaze estimation algorithms require a large amount of data to obtain better results. The collection of gaze data requires specific equipment, and the collection process is cumbersome, tedious and lengthy. Moreover, in some scenarios, like the autism assessment scenario, it is impossible to obtain the gaze training data of autistic children due to their social communication disorders. Therefore, we need to generalize a model trained on public datasets to a new scenario without gaze ground truth labels. In this study, we tackle this problem by leveraging adversarial learning to implement domain adaptation. Besides, we propose an outlier loss to supervise the outputs of the target domain. We test our domain adaptation algorithm on the XGaze-to-MPII domain adaptation task, and achieve a performance improvement of 14.7%.

Autism spectrum disorders is a range of neurodevelopmental conditions primarily characterized by difficulties in social interactions, differences in communication, and presentations of rigid and repetitive behavior. The evidence shows that the functional social behavior of children with autism can be enhanced by early intervention. However, traditional intervention methods meet problems, e.g., assessment results are varied from one clinician to another while sometimes children are lack of interest in intervention. To address these problems, we design a computer-aided motion imitation assessment system based on human pose estimation in this paper. The system is implemented by Unity3D. We recruit 10 people (5 people with imitation ability defect and 5 people without imitation ability defect) participated in the experiment, and the result shows that the system can effectively evaluate the motion imitation ability. Finally, three future development directions of the system are further discussed for better application in autistic early intervention.

Autism Spectrum Disorder (ASD) is a congenital neurodevelopmental disorder, and the number of ASD has been increasing in recent decades worldwide. Early screening is essential for proper treatment and intervention in toddlers with ASD. However, manual early screening methods for ASD are costly and inefficient. Stereotyped behavior is one of the clinical manifestations of ASD toddlers. In this paper, we propose a vision-based action detection network, named OstAD, for response-to-instruction (RTI) protocol to assist professional clinicians with an early screening. Our network adopts a temporal attention branch to aggregate contextual features, and proposes a spatial attention branch to generate local frame-level features of the toddlers. Experimental results demonstrate that the proposed OstAD model can detect typical actions of ASD toddler with mAP 72.6% and 75.9% accuracy, and achieves the excellent results in the RTI screening.

Joint attention is an important skill that involves coordinating the attention of at least two individuals towards an object or event in early child development, which is usually absent in children with autism. Children’s joint attention is an essential part of the diagnosis of autistic children. To improve the effectiveness of autism screening, in this paper, we propose a multi-task facial landmark detection network to enhance the stability of gaze estimation and the accuracy of the joint attention screening result. In order to verify the proposed method, we recruit 39 toddlers aged from 16 to 32 months in this study and build a children-based facial landmarks dataset from 19 subjects. Experiments show that the accuracy of the joint attention screening result is 92.5 $$\%$$ % , which demonstrates the effectiveness of our method.

Using eye tracking technology to explore the underlying processing mechanisms of joint attention in children with autism spectrum disorders (ASD). The experiment selected 32 ASD children and 34 IQ-matched typically developing (TD) children. By freely viewing different hand-up (palm-up, does not respond to intention information; palm-down, grasping action, response intention information) action videos to explore whether hand movements affect joint attention in children with ASD. The results showed that (1) ASD children and TD children had significantly greater dwell time and fixation counts to the post-cued target than non-targets; (2) Hand movements would affect the joint attention of ASD children and TD children. The dwell time and fixation counts were significantly greater in palms down condition than those with palms up. This suggests that children with ASD have joint attention, and this joint attention is based on intentional information.

Children with autism spectrum disorder (ASD) have significant challenges in social interaction. With the rapid development of intelligent robot technology, robot assisted ASD children therapy has penetrated into the intervention of social skills. In this paper, we mainly focus on reviewing the recent development on robot assisted social skills intervention of children with ASD. First, five kinds of robots for ASD children assisted diagnosis and treatment are introduced and the respective advantages and disadvantages are analyzed. Then, five robot assisted social skill intervention scenarios are reviewed and analyzed. Finally, the future research directions on robot assisted ASD children diagnosis and treatment are proposed.

The complex handwriting skill needs reasonable planning along with fine motor skill. However, most children with Autism Spectrum Disorder (ASD) possess motor deficits adversely affecting handwriting. Specifically, it can be divided into extreme finger force, unsmooth handwriting, uncoordinated fingers and other problems, addressable through intervention and robots can provide efficient and engaging ASD intervention environments for children with Autism. The work presented in this paper aims at targeted training of the skills required for writing by developing a multifunctional hand-held robot. The robot is composed of pressure sensor, optical flow sensor, led and other components and its interactive system includes finger strength training game and trajectory training game. Children with autism complete the training by engaging in emotional touch behaviors through an interactive game system. This study conducted a usability study with 8 right-handed healthy volunteers. The results showed significant improvements in finger strength control, fine motor and trajectory control. In particular, the results of trajectory training show that more visual feedback early in training can help them understand and adapt to the game more quickly. It can be expected that the system can be also applied for children with autism for motor training.

This study employed visual tracking technology to explore the effects of a third-person perspective on the fixation condition of social scenarioss in children with autism spectrum disorder (ASD). This study selected 24 ASD children and 24 psychologically age-matched typically developmenting (TD) children as control group, and used eprime 3.0 to present experiment image cosist of social stimuli (a person’s smiling expression) and non-social stimuli (Circumscribed Interests, control objects) by pairing. The results reported that: first, children with autism spend significantly longer looking at the images of social scenarios in the third-person perspective than objects; second, children with autism spend significantly longer looking at image in the third-person perspective than in the first-person perspective; third, when the distractor is Circumscribed Interests, children with ASD spend shorter time to the first fixation of both social scenarios and objects than TD children in the first-person perspective.

Neurodevelopmental disorder refers to behavioral and cognitive impairment during development, which is manifested as significant difficulties in intelligence, motor or social skills. Among these disorders, autism spectrum disorder (ASD), language disorder (LD) and mental retardation (MR) are easily misclassified due to their similar symptoms, which can be very detrimental to later treatment. Therefore, it is important to screen and classify these types of patients correctly. Traditionally, diagnosis of these disorders has relied on the American Psychiatric Research Association’s DSM-5, but these methods require professionals and special assessments to diagnose patients, which can take a lot of time and cost. Nowadays, thanks to the development of deep learning, many researches have used some biological signals (such as electroencephalogram, facial expression and gesture, etc.) for the early diagnosis of neurodevelopmental diseases and have achieved good results. In this work, we proposed a method based on deep learning, which can well distinguish different types of patients only by children’s hand gestures. In order to verify the effectiveness of the method, we conducted a series of experiments on the TASD dataset, and finally the classification accuracy of these three types of patients reached 99.42%. It proves that using only hand gestures is also effective in the screening of ASD, LD, and MR.

Human visual focus is a vital feature to uncover subjects’ underlying cognitive processes. To predict the subject’s visual focus, existing deep learning methods learn to combine the head orientation, location, and scene content for estimating the visual focal point. However, these methods mainly face three problems: the visual focal point prediction solely depends on learned spatial distribution heatmaps, the reasoning process in post-processing is non-learnable, and the learning of gaze salience representation could utilize more prior knowledge. Therefore, we propose a coarse-to-fine human visual focus estimation method to address these problems, for improving estimation performance. To begin with, we introduce a coarse-to-fine regression module, in which the coarse branch aims to estimate the subject’s possible attention area while the fine branch directly outputs the estimated visual focal point position, thus avoiding sequential reasoning and making visual focal point estimation is totally learnable. Furthermore, the human visual field prior is used to guide the learning of gaze salience for better encoding target-related representation. Extensive experimental results demonstrate that our method outperforms existing state-of-the-art methods on self-collected ASD-attention datasets.

This paper proposes a state estimation method of non-cooperative target, which can be used to identify the target satellite motion. Firstly, tumbling motion of the target is analyzed, while dynamic of non-cooperative target is built. Secondly, an estimate method based on least squares method is proposed to identify the kinematic and dynamic parameters. Both of them are based on least squares methods. Thirdly, with parameters estimated, error state Kalman filter is used to estimate the angular velocity and filter attitude of the target at the same time. Finally, a simulation experiment is carried out to verify the effectiveness of the method. Simulation results reveal the method proposed by this paper can identify parameter and estimate motion state accurately, which is meaningful for non-cooperative target capture.

This paper investigates the problem of event-triggered adaptive tracking control for space manipulator systems under pre-determined position constraints. This control scheme aims to overcome external perturbations, reduce the burden of data-transmission, and achieve constrained tracking. Focusing on the constraints of system performance, quadratic Lyapunov functions (QLF) are stitched with a set of asymmetric time-receding horizons (TRH) with fixed settling time, serving as a sufficient condition for the practically prescribed finite-time stability (PPFS) of target plants. By introducing event-triggered conditions, the control signals are transformed into non-periodically updated variables, promoting signaling efficiency while preserving the desired system performance. Complex nonlinearities are integrated and compensated adaptively, providing an ingenious design process and simplifying the construction of the controller. Finally, simulations demonstrate the effectiveness of the proposed scheme.

This paper proposes a compliant strategy for a underactuated gripper to grasp constrained space cooperative target based on the hybrid impedance control. The underactuated gripper is equipped with a differential mechanism and has characteristics of large misalignment tolerance, passive compliant grasp. The strategy consists of two parts: task planning and end effector control algorithm. In the task planning, through the analysis of the contact force and moment caused by the pose misalignment in a single direction, a grasping method of stepwise correcting pose is proposed to avoid the wrong pose correction caused by the complex contact state during the grasping process. In the control algorithm, a six-DOFs hybrid impedance control with variable desired force and variable impedance parameters is designed. The hybrid impedance control algorithm is designed separately in translational direction and rotation direction, to improve the control coupling between the orientation and the position. The underactuated gripper with the compliant strategy can mitigate the impact effect by increasing the interacting time. A set of numerical simulations and capture experiments verify validity of proposed strategy, the correctness of the algorithm and the compliance of the gripper.

The learning-based approach has been proved to be an effective way to solve multi-agent path finding (MAPF) problems. For large warehouse systems, the distributed strategy based on learning method can effectively improve efficiency and scalability. But compared with the traditional centralized planner, the learning-based approach is more prone to deadlocks. Communication learning has also made great progress in the field of multi-agent in recent years and has been be introduced into MAPF. However, the current communication methods provide redundant information for reinforcement learning and interfere with the decision-making of agents. In this paper, we combine the reinforcement learning with communication learning. The agents select its communication objectives based on priority and mask off redundant communication links. Then we use a feature interactive network based on graph neural network to achieve the information aggregation. We also introduce an additional deadlock detection mechanism to increase the likelihood of an agent escaping a deadlock. Experiments demonstrate our method is able to plan collision-free paths in different warehouse environments.

To achieve agile running of a biped robot, dynamic stability, joint coordination, and real-time ability are required. In this paper, a task-space-based controller framework is constructed with a reduced-order 3D-SLIP model. On the top layer, a 3D-SLIP model based planner is employed for center-of-mass trajectory planning. The planner built with optimization for table divided apex state, and a neural network is used to fit the optimized table for real-time planning. On the bottom layer, a task-space-based controller with full-body dynamics is utilized, which solves the quadratic programming for the optimized joint torque in real-time. A 12-DOF biped robot model with a point-foot is used for simulation verification. The simulation result show that stable running and single-cycle apex state change running can achieved with the framework.

The double-tripod multi-loop mechanism (DTMLM) is expected to construct modular system for space applications, such as space capturing, thus requiring good performance, including good dynamics characteristics, kinematics features, and high actuation-transmission efficiency. This paper proposes local and global performance indicators in terms of kinematics and dynamics, by means of concept of the output-array ellipsoid. Then, the kinematics-and dynamics- performance drawings are plotted and then employed to indicate the relationship between the indicators and rod parameters. A multi-objective-optimization procedure is represented to calculate the weight coefficient of the indicators. The diagram of comprehensive evaluation indicators is henceforth plotted. Due to the optimization, $$r=3d_0$$ r = 3 d 0 is better for the DTMLM structures in space applications, which provides strong support in further structure-design process.

In this paper, a flexible rod-driven VGTM for morphing wings is designed. The mechanism contains two parts: flexible parallel mechanism and VGTM. The flexible drive reduces the drives’ number and weight of the mechanism. The VGTM not only reduces the weight of the mechanism but also simplifies the control model. The mechanism can realize multiple deformations including span length, sweep, dihedral and wingtip. It has the advantages of simple structure, light weight, simple control method and many deformation methods. We analyze the degrees of freedom of the mechanism using the screw theory, which demonstrates the deformation ability of the mechanism. The kinematic modeling and control strategy analysis were completed, which demonstrates the feasibility and advantages of this mechanism for morphing wings. And the deformation mode and prototype of the mechanism are also presented in this paper.

Cable-driven parallel robot (CDPR) is widely used in the fields of hoisting and cargo handling. However, it is very easy to be interfered with and restricted by obstacles in the space, which affects the working performance of the robot. This paper takes the reconfigurable cable-driven parallel robot (RCDPR) as the research object and adopts deep reinforcement learning (DRL) to solve the obstacle avoidance planning problem. The model of RCDPR is structured, and kinematic analysis is performed to obtain the state transform function. An improved Soft AC algorithm is employed by expected SARSA and adaptive target values, which enhances the utilization of samples. An environment for RCDPR obstacle avoidance tasks is built, and then the improved Soft AC algorithm is used to train the robot to avoid obstacles in the environment. Finally, the simulation and experimental study of the paths and trajectories generated by the policy neural network are performed to verify the feasibility and effectiveness of the proposed algorithm. The results show that RCDPR can realize autonomous planning and intelligent obstacle avoidance by DRL.

In the paper, a novel modular self-assembling and self-reconfiguring robot named Ant3DBot was proposed, which has many configurations. Ant3DBot consists of four semicircular iron spheroid shells, telescopic legs, and internal magnets that can rotate around the center. Ant3DBot can expand its shells and legs through a single motor, a synchronous belt and compressed springs, which results in two different docking states. Ant3DBot which has the height of 12 cm can traverse obstacles with the height of 8 cm, and pass through a 25° slope in extending configuration. For many unstructured environments, the cooperation of multiple Ant3DBots can reach a target point with simple control. The simulations show the basic ability of a single module to overcome obstacles as well as the cooperative motion of multiple robots. The results demonstrate that the Ant3DBot system has excellent locomotion performance and versatility.

The robot dexterous hand is a highly flexible and complex end-effector. In response to the complex drive transmission mechanical structure of the traditional humanoid dexterous hand and the difficulty of assembly and maintenance control, etc., a modular linkage-driven robotic finger is designed in this paper based on a linkage drive mechanism. It has two degrees of freedom (two joints) and is compact in structure, low in cost and simple in assembly and maintenance. Compared with underactuated fingers, it is featured with greater dexterity and a stronger adaptive grasping ability. The modular robotic finger kinematics model is established and analyzed. Finally, simulation based on ROS and the experiment based on a finger prototype are constructed. A straight line and a circular arc trajectory is designed to verify the performance of the proposed kinematics method and the feasibility of the modular robotic finger mechanism. Experimental results show that the proposed kinematics method has high accuracy and the designed modular robotic finger structure is reliable.

Aiming at leakage, crystallization and deformation of RIC (In-Core Instrumentation) system, the thermocouple is mainly used to measure the outlet temperature of the reactor core coolant in the RIC system, which plays an important role in the condition monitoring of the internal equipment in the nuclear reactor core. To prevent serious nuclear accidents and reduce the radiation to maintenance personnel, a thermocouple component robotic replacement system is built in the nuclear reactor. This paper analyzes the increased demand for thermocouple components replacement. According to the demand, the context plans the specific workflow of the robot, designs three dexterous end effectors, adopts an incremental master-slave mapping algorithm and completes the overall design of the robotic system. Finally, an experimental setup is built to test the operation process of the robotic system, which proved the system can complete the assembly replacement requirements. The system lays the foundation for the development and engineering application on next step.

Dexterous robotic hands can not only work in dangerous situations like industrial processing, but also play a full role in daily lives such as artificial hands for the disabled, nursing home services and other fields. And smart materials driven dexterous hands are lightweight, highly compliant and have low risk of failure. However, the poor stiffness and low driving force limit their practical applications. In this paper, we propose a design of the dexterous hand driven by dielectric elastomer actuator (DEA), and incorporate origami structure to improve its stiffness and bearing capacity. Then, an analysis model for the DEA and knuckle is presented. Through this analysis model, not only the bending curvature, tip displacement and driving force of DEA, but also the compressive displacement, bending angle and driving force of the knuckle can be obtained after determining the design parameters (dimensions and materials) and input voltage. And the curvature model of DEA is verified experimentally. This paper lays a structural and theoretical foundation for subsequent research of the dexterous hand, showing that the combination of DEA with origami structure has great prospects in the field of dexterous hands.

Learning a policy from sparse rewards is a main challenge in reinforcement learning (RL). The best solutions to this challenge have been via sample inefficient model-free RL algorithms. Model-based RL algorithms are known to be sample efficient but few of them can solve sparse settings. To address these limitations, we present PlanQ, a sample efficient model-based RL framework that resolves sparse reward settings. PlanQ leverages Q-values that encode long-term values and serve as a richer feedback signal to actions than immediate rewards. As such, PlanQ scores rollout returns from its learned model with returns containing Q-values. We verify the efficacy of the approach on robot manipulation tasks whose difficulties range from simple to complex. Our experimental results show that PlanQ enhances performance and efficiency in sparse reward settings.

This paper presents some of the recent progress of an underwater robotic avatar. The manipulation system of the avatar is enhanced in the terms of compliance, including the arm and the gripper. A rigid-foldable mechanism is applied to develop a compliant robotic arm that has the advantages of light weight, compactness, and expandability. By proposing a shape memory alloy-based module, the adjustable grasping stiffness of the gripper is achieved. Moreover, A human-robot shared control scheme is applied to reduce the burden on the human operator and enable high-level intelligent human-robot collaboration. Preliminary experimental results illustrate that the proposed components can meet the expected performance requirements, providing sufficient prior experience for future total integration of the underwater robotic avatar system.

Bimanual activities like coffee stirring, which require coordination of dual arms, are common in daily life and intractable to learn by robots. Adopting reinforcement learning to learn these tasks is a promising topic since it enables the robot to explore how dual arms coordinate together to accomplish the same task. However, this field has two main challenges: coordination mechanism and long-horizon task decomposition. Therefore, we propose the Mixline method to learn sub-tasks separately via the online algorithm and then compose them together based on the generated data through the offline algorithm. We constructed a learning environment based on the GPU-accelerated Isaac Gym. In our work, the bimanual robot successfully learned to grasp, hold and lift the spoon and cup, insert them together and stir the coffee. The proposed method has the potential to be extended to other long-horizon bimanual tasks.

In this work, a novel model-free robotic residual errors compensation method is proposed based on the information-diffusion-based dataset enhancement (ID-DE) and the Gradient-Boosted Decision Trees (GBDT). Firstly, the dataset enhancement method is developed by utilizing the normal membership function based on the information diffusion technology. Then, merging it with multiple GBDTs, the multi-output residual errors learning model (ID-GBDTs) is constructed, and the grid search is used to determine the optimal hyper-parameters to accomplish the accurate prediction of residual errors. Finally, the compensation of robotic residual errors is realized by using the calibrated kinematic model. Experiments show that ID-DE can significantly improve the generalization ability of various learning models on the few-shot dataset. The R-squared of ID-GBDTs is improved from 0.58 to 0.77 along with the MAE decreased from 0.23 to 0.16, compared to original GBDT. Through the compensation of the residual errors, the mean/maximum absolute positioning error of the UR10 robot are optimized from 4.51/9.42 mm to 0.81/2.65 mm, with an accuracy improvement of 82.03%.

Aiming at the shortcomings of current mobile robot integrated power drive wheels, in this paper, an integrated power wheel module suitable for large industrial Automated Guided Vehicle (AGV)/mobile robots is proposed. Different from the traditional structure, this paper makes innovations in the design of the motor, the topology of the drives and the integrated structure. In order to improve the torque density, an axial flux motor is adopted, and the structure of the reducer and the motor in series is used, which greatly reduces the size of the motor. In terms of drive technology, the winding with common DC bus including dual-inverter circuit structure improves the fault-tolerant capability.

Human following is an essential function in many robotic systems. Most of the existing human following algorithms are based on human tracking algorithms. However, in practical scenarios, the human subject might easily disappear due to occlusions and quick movements. In order to solve the problem of occlusion, this paper proposed a classification-based human following framework. After using a pre-trained MobileNetV2 model to detect the human subjects, the robot will automatically train a classification model to identify the target person. In the end, the robot is controlled by some rule-based motion commands to follow the target human. Experimental results on several practical scenarios have demonstrated the effectiveness of the algorithm.

Traditional multi-objective reinforcement learning problems pay attention to the expected return of each objective under different preferences. However, the difference in strategy in practice is also important. This paper proposes an algorithm Multi-objective RL with Preference Exploration (MoPE), which can cover the optimal solutions under different objective preferences as much as possible with only one trained model. Specifically, the coverage of the optimal solution is improved by exploring the preference space in the sampling stage and reusing samples with similar preferences in the training stage. Furthermore, for different preference inputs, a variety of diversity strategies that conform to the preference can be generated by maximizing the mutual information of preference and state based on a method of information theory. Compared with the existing methods, our algorithm can implement more diverse strategies on the premise of ensuring the coverage of the optimal solution.

In view of the shortcomings of the existing service robot manipulator, such as low flexibility and small load, the tendon-sheath transmission is creatively applied to design a 7-DOF robotic arm. The joint and the drive module are separated by the tendon-sheath transmission and designed individually. Considering the transmission characteristics of the gear reducer and the tendon-sheath, position transmission model of the joint is built. Proportional-integral-differential (PID) controller and iterative learning controller are designed for position tracking control based on double encoders. Position control experiments of the elbow joint are carried out based on the constructed physical prototype. The experimental results show that compared with PID controller, iterative learning controller can effectively reduce the position tracking error and improve the position control accuracy. Applying this transmission method to the manipulator can improve its cost performance ratio, which provides the possibility for the large-scale application of the manipulator in more scenes.

Aiming at the difficulty of pelvic motion control and knee bending during rehabilitation training for patients with limb motor dysfunction, a walking training robot with pelvic motion assistance function was proposed. First, according to the needs of clinical rehabilitation training and the design requirements of rehabilitation robot, the overall structure of a pelvic-assisted walking training robot (PAWTR) with left-right symmetrical arrangement was innovatively designed. The unilateral pelvic-assisted walking unit (PAWU) adopted a single-drive double-exit structure. Secondly, the kinematics model of PAWU was established, and the mechanism parameters were analyzed and determined, which laid the foundation for the robot motion planning. Finally, the robot motion planning methods in the single-leg swing and double-leg swing training modes were proposed, and the influence of different walking cycle durations was studied. The simulation verified that the robot motion planning was feasible and could meet the requirements of walking rehabilitation training.

This paper describes an impedance control strategy based on model reference adaptation in unstructured environment, aimed at the uncertainty of the environmental stiffness and the unknown of the dynamic change of the environmental position during force tracking. First, the contact force model between the robot and the environment is established, and the environmental stiffness is identified through the BP (back propagation) neural network; then, the simulink-adams co-simulation model of dynamic-based adaptive force control is established. The change of the contact force adjusts the parameters of the impedance model online adaptively, which is used to compensate for the unknown dynamic change of the environment; finally, the simulation results show that the strategy can achieve a good force control effect, and the control method has strong robustness It can increase the reliability of the system, and is suitable for robotic arm force interaction scenarios in a location environment.

This study proposes an innovative fall detection and alarm system for the elderly in the family environment based on deep learning. The overall cost of hardware development is a camera and an edge device like a Raspberry PI or an old laptop that can detect and alert users to falls without touching the user’s body. The development idea of the system is as follows: 1. Collect the pictures of falling and normal states under different conditions; 2. The improved lightweight SSD-Mobilenet object detection model is used to train the data set and select the optimal weight; 3. Optimal results are deployed on a Raspberry PI 4B device using a lightweight inference engine Paddle Lite. The mean Average Precision of the best model is 92.7%, and the detection speed can reach 14FPS (Frames Per Second) on the development board. When the camera detects that someone has fallen for 10 s, the compiled script sends an alert signal to the default guardian’s email via the Mutt email program on Linux. The experimental results show that the fall detection system achieves satisfactory detection accuracy and comfort.

In order to improve the efficiency of enterprises in the sorting task of packaging boxes and reduce the labor intensity of workers, a fast and efficient detection method based on image processing is proposed in this study. This research mainly involves pose estimation of the packaging box, as well as solving the transformation relationship from the pose of the packaging box in the camera coordinate system to the base coordinate system of the manipulator. SIFT method is used to obtain packaging feature points, FLANN method is used for feature point matching, and EPnP method is used to solve the pose of the box. This study uses nine-point calibration method to solve the transformation relationship between the base coordinate system of manipulator and camera coordinate system. It can be seen from the test results that the method used in this study has achieved satisfactory results by weighing the two indicators of detection accuracy and speed.

Large wingspan birds inspire the studies about the bionic fold-able flapping-wing robot. To explore the influencing factors of the bionic ornithopter’s aerodynamic characteristics, two bionic fold-able flapping-wing flapping mechanisms were established to simulate the flight motion of birds. We present two fold-able flapping-wing structural design methods with parameters: inner wing flapping and outer wing folding angles. A simulation analysis is carried out for these structures based on XFlow, and the influence on lift and thrust is explored with different flapping frequencies and air velocities. The results show that the lift and drag of the ornithopter increase with air velocity, and the thrust increases with the rise of flapping frequency. Furthermore, the comparisons indicate that an asymmetric flapping mode of fold-able wing structure has better aerodynamic characteristics. The critical contribution of this paper is that we propose helpful structural design guidance for flapping-wing robots.

A robotic hand with 19 joints is designed to make the mechanical hand lighter and more anthropomorphic. Inspired by the human flexor tendon and sheath, tendon-sheath transmission is applied to drive the finger joint, which decouples the motion of the joints and achieves the postposition of the drive motor. The configuration of the robotic hand is determined by referring to the joints of a human hand; furthermore, a joint mechanism and a drive structure are designed. The flex sensor and the changeable proportion mapping algorithm are applied, and the tracking control of grasping is achieved. Finally, a mechanical hand prototype is built to conduct gesture experiments and grasping control experiments. According to the experimental results, the designed hand has high motion flexibility, and the cooperation of the fingers achieves the effective grasping of various objects. The application of tendon-sheath transmission to the mechanical hand is feasible, and the research context has certain theoretical value and practical significance for the technical development and social application of anthropomorphic multijoint robotic hands.

The design of a cable-driven interactive rehabilitation device with 3D trajectory tracking and force feedback is presented in this paper. This device is designed for the upper limb active training, including muscle strengthening and full range of 3D space motion training. Unlike the traditional end-effector robot, this device only offers tensile force to the user by grasping the handle, which attached to the end of the cable. The force value, force direction and handle position are real-time monitored by three force sensors and an encoder. This enables more interesting interactive training between the user and the device. The mechanical design and control system design are presented in detail. The motion space of the device and the human model are analyzed. The PID force controller was designed to keep the tensile force accurately tracking given trajectories. Experiment with different PID parameters was carried out and the results show that the designed PID controller has relatively optimal control performance, with sine and square wave tracking errors are respectively −0.018 ± 0.56 N and −0.11 ± 3.45 N. The proposed device is potentially to provide physical fitness training, in addition to the routine training therapy in daily life.

Lower Limb Exoskeleton (LLE) has received considerable interests in strength augmentation, rehabilitation and walking assistance scenarios. For walking assistance, the LLE is expected to have the capability of recognizing the motor intention accurately. However, the methods for recognizing motor intention base on ElectroEncephaloGraphy (EEG) can not be directly used for recognizing the motor intention of human lower limbs, because it is difficult to distinguish left and right limbs. This paper proposes a human-exoskeleton interaction method based on EEG and ElectroMyoGrams (EMG)-Hierarchical Recognition for Motor Intention (HRMI). In which, the motor intention can be recognized by the EEG signal, and supplemented by EMG signals reflecting motor intention, the exoskeleton can distinguish the left and right limbs. An experimental platform is established to explore the performance of the proposed method in real life scenario. Ten healthy participants were recruited to perform a series of motions such standing, sitting, walking, and going up and down stairs. The results shown that the proposed method is successfully applied in real life scenarios and the recognition accuracy of standing and sitting than others.

The design of parallel mechanisms with multi-rotational degrees of freedom (DOFs), especially with large orientation workspace, is still a tough task. This paper presents the design process of new multi-DOF parallel mechanisms with large orientation workspace, which can be used in haptic application. First, based on the configurable design concept, two configurable platforms with two or three rotational DOFs are designed. By means of the designed platforms and Lie group theory, a series of parallel mechanisms with two rotational and three translational (2R3T) DOFs or three rotational and three translational (3R3T) DOFs are developed. According to the synthesized parallel mechanisms, a haptic device with reconfigurable ability is proposed. The analysis of rotational capability is carried out in terms of one of the synthesized parallel mechanisms. The results reveal that resorting to actuation redundancy, the studied PM achieves the design requirement of large orientation workspace.