Robotics Research

The ability to move between air and water with miniature robots would allow distributed water sampling and monitoring of a variety of unstructured marine environments, such as coral reefs and coastal areas. To enable such applications, we are developing a new class of aerial-aquatic robots, called Aquatic Micro Aerial Vehicles (AquaMAVs), capable of diving into the water and returning to flight. One of the main challenges in the development of an AquaMAV is the provision of sufficient power density for take-off from the water. In this paper, we present a novel system for powerful, repeatable aquatic escape using acetylene explosions in a 34 g water jet thruster, which expels water collected from its environment as propellant. We overcome the miniaturisation problems of combustible fuel control and storage by generating acetylene gas from solid calcium carbide, which is reacted with enviromental water. The produced gas is then combusted in air in a valveless combustion chamber to produce over 20 N of thrust, sufficient to propel small robots into the air from water. The system for producing combustible gases from solid fuels is a very compact means of gas storage, and can be applied to other forms of pneumatic actuation and inflatable structure deployment.

We present a nonlinear hover controller for a small flying wing tailsitter vehicle, which enables recovering to hover from a large set of initial conditions. The proposed attitude control law is obtained by solving an optimal control problem, with the objective of correcting large attitude errors by turning primarily around the vehicle’s strongly actuated axis. Solutions for a set of initial attitudes are precomputed and stored in a lookup table. For each controller update, the optimal inputs are read from this table, and applied to the system in an MPC-like manner. Simulation results indicate that this control method is able to perform recoveries to hover from any initial attitude, given that the initial velocity of the vehicle is below a certain limit. Further, the performance of the control strategy is demonstrated on a small tailsitter vehicle in the ETH Zurich Flying Machine Arena.

This paper addresses the problem of autonomous quadrotor navigation through a previously-mapped indoor area. In particular, we focus on the case where a user walks through a building and collects images. Subsequently, a visual map of the area, represented as a graph of linked images, is constructed and used for automatically determining visual paths (i.e., sequences of images connecting the start to the end image locations specified by the user). The quadrotor follows the desired path by iteratively (i) determining the desired motion to the next reference frame, (ii) controlling its roll, pitch, yaw-rate, and thrust, and (iii) appropriately switching to a new reference image. For motion estimation and reference-image switching, we concurrently employ the results of the 2pt and the 5pt RANSAC to distinguish and deal with both cases of sufficient and insufficient baseline (e.g., rotation in place). The accuracy and robustness of our algorithm are evaluated experimentally on two quadrotors navigating along lengthy corridors, and through tight spaces inside a building and in the presence of dynamic obstacles (e.g., people walking).

Insect-scale micro-air vehicles (MAVs) require careful consideration of the size, weight and power for each component. The inherent instability of the system, exacerbated by the faster dynamics that result from increasing angular accelerations with decreasing scale, requires high bandwidth sensing to maintain stable flight. The Harvard RoboBee is the first MAV under 100 mg to demonstrate controlled flight using external motion capture cameras to measure the position and orientation of the vehicle during flight. Prior research into onboard sensing has demonstrated several sensors that provide sufficiently high-bandwidth and low-latency feedback to stabilize the attitude of the robot. To achieve autonomous flight, the vehicle needs to sense its attitude, altitude, and either lateral position or velocity. Here we build on previous work by incorporating a sensor that is size- and power-compatible with insect-scale flight, capable of estimating distance by measuring the time-of-flight of an infrared laser pulse. This sensor has sufficiently low latency to allow the robot to maintain constant altitude over multiple flight experiments. This work on onboard altitude control represents the latest results in achieving autonomous control and visually-guided flight.

Autonomous construction with aerial vehicles has great potential for in-situ repair and construction in hard-to-access areas. In this paper, we present and demonstrate a mechanism by which a team of autonomous nano aerial vehicles construct a multi-element tensile structure between anchor points in an irregular environment, such as a natural woodland. Furthermore, we demonstrate potential applications of such a structure to enable long-term position holding of aerial vehicles that are otherwise extremely limited in terms of available flight time due to energy constraints. To demonstrate the effectiveness of this mechanism, we develop the mechanical and electronic designs of two payload packages for attachment to nano quadrotor robots with a total integrated mass of only 26 g per robot, and we present the trajectory planning and control algorithms required to enable robust execution of the construction scheme.

We apply a combination of linear time varying (LTV) Kalman filtering and nonlinear contraction tools to the problem of simultaneous mapping and localization (SLAM), in a fashion which avoids linearized approximations altogether. By exploiting virtual synthetic measurements, the LTV Kalman observer avoids errors and approximations brought by the linearization process in the EKF SLAM. Furthermore, conditioned on the robot position, the covariances between landmarks are fully decoupled, making the algorithm easily scalable. Contraction analysis is used to establish stability of the algorithm and quantify its convergence rate. We propose four versions based on different combinations of sensor information, ranging from traditional bearing measurements and radial measurements to optical flows and time-to-contact measurements. As shown in simulations, the proposed algorithm is simple and fast, and it can solve SLAM problems in both 2D and 3D scenarios with guaranteed convergence rates in a full nonlinear context.

Vision-based localization systems rely on highly-textured areas for achieving an accurate pose estimation. However, most previous path planning strategies propose to select trajectories with minimum pose uncertainty by leveraging only the geometric structure of the scene, neglecting the photometric information (i.e, texture). Our planner exploits the scene’s visual appearance (i.e, the photometric information) in combination with its 3D geometry. Furthermore, we assume that we have no prior knowledge about the environment given, meaning that there is no pre-computed map or 3D geometry available. We introduce a novel approach to update the optimal plan on-the-fly, as new visual information is gathered. We demonstrate our approach with real and simulated Micro Aerial Vehicles (MAVs) that perform perception-aware path planning in real-time during exploration. We show significantly reduced pose uncertainty over trajectories planned without considering the perception of the robot.

As a visual-inertial navigation system (VINS) becomes prevalent thanks to recent advancements in cameras and inertial sensors, optimal sensor fusion algorithms are demanding. In this paper, we introduce a new optimal-state-constraint (OSC)-EKF for VINS, which performs tightly-coupled visual-inertial sensor fusion over a sliding window of poses only (i.e., without including features in the state vector), and thus has complexity independent of the size of the environment. The key idea of the proposed OSC-EKF is to design a novel measurement model that utilizes all feature measurements available within the sliding window and derives probabilistically optimal constraints between poses while without estimating these features as part of the state vector. To this end, for each sliding window, we perform structure and motion using only the available camera measurements and subsequently marginalize out the structure (features) to obtain the optimal motion constraints that will be used in the EKF update. The proposed approach is validated in the proof-of-concept, real-world experiments.

We present a unified formulation for describing the dynamics of a new class of aquatic multi-body soft robots. This formulation accounts for the continuum hyperelastic nature of the vehicles and for the interaction with the dense fluid across which they dwell. We start by introducing the highly unconventional design concept of a soft underwater vehicle inspired by the octopus capable of swimming, manipulating and crawling. The dynamics of the robot, which consists of a multi-limbed continuum of elastomeric materials, is extremely complex to account for with conventional modelling tools. Hence a Cosserat based formalism where a Reissner shell model and a finite-strain beam formulation are joined is conceived which lends itself to the description of the highly non linear dynamics of this new family of vehicles in a dense fluid.

Object handover is a basic, but essential capability for robots interacting with humans in many applications, e.g., caring for the elderly and assisting workers in manufacturing workshops. It appears deceptively simple, as humans perform object handover almost flawlessly. The success of humans, however, belies the complexity of object handover as collaborative physical interaction between two agents with limited communication. This paper presents a learning algorithm for dynamic object handover, for example, when a robot hands over water bottles to marathon runners passing by the water station. We formulate the problem as contextual policy search, in which the robot learns object handover by interacting with the human. A key challenge here is to learn the latent reward of the handover task under noisy human feedback. Preliminary experiments show that the robot learns to hand over a water bottle naturally and that it adapts to the dynamics of human motion. One challenge for the future is to combine the model-free learning algorithm with a model-based planning approach and enable the robot to adapt over human preferences and object characteristics, such as shape, weight, and surface texture.

The ability to efficiently move in complex environments is a fundamental property both for animals and for robots, and the problem of locomotion and movement control is an area in which neuroscience and robotics can fruitfully interact. Animal locomotion control is in a large part based on spinal cord circuits that combine reflex loops and central pattern generators (CPGs), i.e. neural networks capable of producing complex rhythmic or discrete patterns while being activated and modulated by relatively simple control signals. These networks located in the spinal cord for vertebrate animals are modulated by descending control signals and interact with the musculoskeletal system for generating rich motor behaviors. This paper presents how numerical models and robots can be used to explore the interplay of these four components (CPGs, reflexes, descending modulation, and musculoskeletal system). Going from lamprey to human locomotion, a series of models are presented that tend to show that the respective roles of these components have changed during evolution with a dominant role of CPGs in lamprey and salamander locomotion, and a more important role for sensory feedback and descending modulation in human locomotion. Interesting properties for robot locomotion control are also discussed.

Intraoperative magnetic resonance image (MRI)-guided neurosurgical procedure is receiving much attention due to the use of real-time image feedback instead of pre-operative images when resecting the tumor. We envision a real-time MR image-guided robotic neurosurgery that utilizes a dexterous meso-scale surgical robot that can work in tight spaces. In this work, we introduce an MR-compatible robotic platform for a spring-based prototype of the minimally invasive neurosurgical intracranial robot (MINIR-II). The robot consists of an outer spring and an inner interconnected spring that has three segments, each of which has two degrees of freedom (DoFs). Each joint of the robot is actuated by an antagonistic pair of shape memory alloy (SMA) spring actuators with integrated water cooling modules. The proposed water-based cooling strategy is designed to improve the cooling rate and thus the actuation bandwidth of SMA springs so that the neurosurgical robot can be operated at sufficiently high bandwidth. We characterized our cooling module integrated SMA springs based on several parameters including the current supplied, water flow rate, SMA pre-strain, gauge pressure of the compressed air, and motion amplitude. We developed a vision-based experimental setup to perform the characterization experiments and optimized the actuator performance in terms of its actuation bandwidth. We commanded the base and middle segments of the robot to follow a series of step input references to verify the improved actuation bandwidth of the antagonistic SMAs. Finally, we performed experiments to allow continuous and coordinated motion between the base and middle segments to verify the robot’s independent joint controllability and motion repeatability.

This paper presents the design, fabrication, and experimental characterization of variable stiffness inflatable structures for soft pneumatic supernumerary robotic (SR) fingers. These novel pneumatic SR fingers consist of inflatable, rigidizable finger phalanges and variable stiffness pneumatic bending actuators which are designed and manufactured using soft robot fabrication methods. The buckling conditions of the phalanges are predicted using a simplified deformation model and validated by empirical data to allow the prediction load bearing capacity, and the mechanical behavior of the bending actuators is empirically characterized for stiffness tuning capability. Experimental testing of a wearable, clinically-focused pneumatic SR grasp assist device demonstrates the feasibility of using variable stiffness pneumatic structures to produce grasp synergies without the need for complicated, high power mechanisms or precise, low-level motion control.

We propose a method that allows a motion planning algorithm to imitate the behavior of expert users in deformable environments. For instance, a surgeon inserting a probe knows intuitively which organs are more sensitive than others, but may not be able to mathematically represent a cost function that governs his or her motion. We hypothesize that the relative sensitivities of deformable objects are encoded in the expert’s demonstrated motion, and present a framework which is able to imitate an expert’s behavior by learning a sensitivity-based cost function under which the expert’s motion is optimal. Our framework consists of three stages: (1) Automatically generating demonstration tasks that prompt the user to provide informative demonstrations through an active learning process; (2) Recovering object sensitivity values using an Inverse Optimal Control technique; and (3) Reproducing the demonstrated behavior using an optimal motion planner. We have tested our framework with a set of 5DoF simulated and 3DoF physical test environments, and demonstrate that it recovers object parameters suitable for planning paths that imitate the behavior of expert demonstrations. Additionally, we show that our method is able to generalize to new tasks; e.g. when a new obstacle is introduced into the environment.

To take full advantage of the flexibility of a modular robot system, users must be able to create and verify new configurations and behaviors quickly. We present a design framework that facilitates rapid creation of new configurations and behaviors through composition of existing ones, and tools to verify configurations and behaviors as they are being created. New configurations are created by combining existing sub-configurations, for example combining four legs and a body to create a walking robot. Behaviors are associated with each configuration, so that when sub-configurations are composed, their associated behaviors are immediately available for composition as well. We introduce a new motion description language (Series-Parallel Action Graphs) that facilitates the rapid creation of complex behaviors by composition of simpler behaviors. We provide tools that automatically verify configurations and behaviors during the design process, allowing the user to identify problems early and iterate quickly. In addition to verification, users can evaluate their configurations and behaviors in a physics-based simulator.

Designing complex machines such as robots often requires multiple iterations of design and prototyping. Folding has recently emerged as a method to both simplify fabrication and accelerate assembly of such machines. However, the robots so far produced by folding have often been made of thin, flexible materials that limit their size and strength. We introduce a folding-based fabrication process that uses thick materials layered with flexible film to enable folding while maintaining high stiffness in the folded structure. We use this process to fabricate multiple solid bodies, as well as two hexapods, one of which can carry up to 2.50 kg payloads. Each folded structure took less than 3 h to construct. Our results indicate that folding using thick materials can be a viable method for rapidly fabricating and prototyping larger and sturdier robots.

Designing a continuum robot’s geometry, sensing its shape/state in space, and planning collision-free trajectories that meet the needs of an application were initially thought of as decoupled problems for continuum robots. However, a body of literature is beginning to emerge showing advantages in solving various combinations of two of these three problems simultaneously. In this paper we argue that all three of these problems are fundamentally connected for continuum robots, that the connection can be analyzed using statistical state estimation, and that considering the three problems simultaneously can lead to better overall solutions. We provide examples for concentric-tube continuum robots.

In this paper we present a novel approach for synthesizing and optimizing both positions and forces in force closure grasps. This problem is a non-convex optimization problem in general since it involves constraints that are bilinear; in particular, computing wrenches involves a bilinear product between grasp contact points and contact forces. Thus, conventional approaches to this problem typically employ general purpose gradient-based nonlinear optimization. The key observation of this paper is that the force closure grasp synthesis problem can be posed as a Bilinear Matrix Inequality (BMI), for which there exist efficient solution techniques based on semidefinite programming. We show that we can synthesize force closure grasps on different geometric objects, and by maximizing a lower bound of a grasp metric, we can improve the quality of the grasp. While this approach is not guaranteed to find a solution, it has a few distinct advantages. First, we can handle non-smooth but convex positive semidefinite constraints, which can often be important. Second, in contrast to gradient-based approaches we can prove infeasibility of problems. We demonstrate our method on a 15 joint robot model grasping objects with various geometries. The code is included in https://github.com/RobotLocomotion/drake.

This paper proposes a new approach to using machine learning to detect grasp poses on novel objects presented in clutter. The input to our algorithm is a point cloud and the geometric parameters of the robot hand. The output is a set of hand poses that are expected to be good grasps. There are two main contributions. First, we identify a set of necessary conditions on the geometry of a grasp that can be used to generate a set of grasp hypotheses. This helps focus grasp detection away from regions where no grasp can exist. Second, we show how geometric grasp conditions can be used to generate labeled datasets for the purpose of training the machine learning algorithm. This enables us to generate large amounts of training data and it grounds our training labels in grasp mechanics. Overall, our method achieves an average grasp success rate of 88% when grasping novels objects presented in isolation and an average success rate of 73% when grasping novel objects presented in dense clutter. This system is available as a ROS package at http://wiki.ros.org/agile_grasp.

Brain-Computer Interfaces are promising technologies that can improve Human-Robot Interaction, especially for disabled and impaired individuals. Non-invasive BCI’s, which are very desirable from a medical and therapeutic perspective, are only able to deliver noisy, low-bandwidth signals, making their use in complex tasks difficult. To this end, we present a shared control online grasp planning framework using an advanced EEG-based interface. Unlike commonly used paradigms, the EEG interface we incorporate allows online generation of a flexible number of options. This online planning framework allows the user to direct the planner towards grasps that reflect their intent for using the grasped object by successively selecting grasps that approach the desired approach direction of the hand. The planner divides the grasping task into phases, and generates images that reflect the choices that the planner can make at each phase. The EEG interface is used to recognize the user’s preference among a set of options presented by the planner. The EEG signal classifier is fast and simple to train, and the system as a whole requires almost no learning on the part of the subject. Three subjects were able to successfully use the system to grasp and pick up a number of objects in a cluttered scene.

In this paper, we propose a new balancing strategy for musculoskeletal humanoids by using their redundant musculoskeletal structures. This strategy is based on the idea of muscle Zero Moment Point(ZMP) and involves the use of a balance stabilizer utilizing the spine. The muscle ZMP is a stabilization indicator instead of a normal ZMP that is computed from 6DOF force sensors installed on robots’ foot. In order to compute the muscle ZMP, we use the joint torques obtained from muscle tensions. The spine stabilizer compensates for the COG displacement of the whole-body by utilizing the spine movements. Further, we confirm the effectiveness of the proposed strategy by demonstrating several balancing motions of Kenshiro, a musculoskeletal humanoid.

In unstructured environments, grasping systems should cope with a wide range of object and environment variations, across size, shape and pose, friction and mass, visual occlusions and shadows, robot control inaccuracy, and many other factors. This paper proposes a framework for analyzing the sources of variations in grasping tasks as a way to understand grasping system performance. The concomitant design approach starts with a collection of basis grasps, each a specific arrangement of the fingers on a specific object. Next, we use motion sequences, sensing, and passive mechanics to make these grasps robust to variations in objects, sensing, and control. We then analyze each grasp’s robustness to local variation to determine the basin of attraction, the range of variation it can tolerate while still achieving a good grasp. Finally, we treat this basin of attraction as a variation budget that can be distributed across subsystems to inform system tradeoffs between object variation, perception errors, and robot inaccuracies. The principle advantage is that within the context of specific grasps, the effects of local variations can be understood and quantified, and therefore compared across disparate approaches.

Fractional order calculus—a generalization of the traditional calculus to arbitrary order differointegration—is an effective mathematical tool that broadens the modeling boundaries of the familiar integer order calculus. Fractional order models enable faithful representation of viscoelastic materials that exhibit frequency dependent stiffness and damping characteristics within a single mechanical element. We propose the use of fractional order models/controllers in haptic systems to significantly extend the type of impedances that can be rendered using the integer order models. We study the effect of fractional order elements on the coupled stability of the overall sampled-data system. We show that fractional calculus generalization provides an additional degree of freedom for adjusting the dissipation behavior of the closed-loop system and generalize the well-known passivity condition to include fractional order impedances. Our results demonstrate the effect of the order of differointegration on the passivity boundary. We also characterize the effective impedance of the fractional order elements as a function of frequency and differointegration order.

For the past forty years, design of robotic wrists in the robot industry has been dominated by a serial kinematics architecture, which parameterizes the end-effector orientation space by Euler angles. Such a design suffers from stationary (or dead-centre) configurations, as well as a weak third axis due to gear train backlash. It was once believed that the study of parallel kinematics mechanisms could result in viable alternatives overcoming the shortcomings of serial wrists. However, this did not happen, probably due to the limited workspace, complex kinematics, and inherent singularities characterizing parallel architectures. In this paper, we propose a novel class of serial-parallel 3-DoF robotic wrists, based on a particular geometry usually found in constant-velocity (CV) shaft couplings. The theory of CV couplings originated with Myard’s study and culminated with Hunt’s work. We have gone one step further, by fully decrypting and completing Hunt’s development using symmetric space theory. The latter allows us to provide an easy-to-follow procedure for synthesizing a unique type of parallel wrists with interconnections. Such novel wrists entail analytic direct and inverse kinematic analyses, and their singularities can be easily identified using the so-called half-angle property, which holds for all symmetric subspaces of the special Euclidean group. By conveniently choosing geometric parameters, the proposed wrists can achieve a singularity-free pointing cone of $$180^\circ $$180∘, in addition to an unlimited rolling.

This paper presents a taxonomy of benchmark manipulation tasks for service robots. Our contributions are threefold: (1) A review of relevant literature regarding manipulation tests in the robotics domain and related fields, such as physical therapy, assistive technologies and prosthetics. (2) Guidelines to design useful testing protocols to evaluate manipulation performance. (3) A proposed general taxonomy of benchmark manipulation tasks and sample tests per each class.

Among the most promising field of applications of wearable robotics there are the rehabilitation and the support in activities of daily living (ADL) of impaired people. In this paper, we propose two possible designs of a robotic extra-finger, the Robotic Sixth Finger, for grasping compensation in patients with reduced hand mobility, such as post-stroke patients. The idea is to let the patients be able to grasp an object by taking advantage of the wearable device worn on the paretic limb by means of an elastic band. The Robotic Sixth Finger and the paretic hand work jointly to hold an object. Adding a robotic opposing finger is a promising approach that can significantly improve the grasping functional compensation in different typologies of patients during everyday life activities.

We investigate the problem of cooperative multi-robot planning in unknown environments, which is important in numerous applications in robotics. The research community has been actively developing belief space planning approaches that account for the different sources of uncertainty within planning, recently also considering uncertainty in the environment observed by planning time. We further advance the state of the art by reasoning about future observations of environments that are unknown at planning time. The key idea is to incorporate within the belief indirect multi-robot constraints that correspond to these future observations. Such a formulation facilitates a framework for active collaborative state estimation while operating in unknown environments. In particular, it can be used to identify best robot actions or trajectories among given candidates generated by existing motion planning approaches, or to refine nominal trajectories into locally optimal trajectories using direct trajectory optimization techniques. We demonstrate our approach in a multi-robot autonomous navigation scenario and show that modeling future multi-robot interaction within the belief allows to determine robot trajectories that yield significantly improved estimation accuracy.

This paper describes a holistic method for automatically synthesizing controllers for a team of robots operating in an environment shared with other agents. The proposed approach builds on recent advances in Reactive Mission Planning using Linear Temporal Logic, and Local Motion Planning using convex optimization. A local planner enforces the dynamic constraints of the robot and guarantees collision avoidance in 2D and 3D workspaces. A reactive mission planner takes a high-level specification that captures complex motion sequencing, and generates a correct-by-construction controller guaranteed to achieve the specified behavior and be reactive to sensor events. If there is no controller that fulfills the specification because of possible deadlock in the local planner, a minimal set of human-readable assumptions is generated as a certificate of the conditions on deadlock where the task is guaranteed. This is truly a synergistic method: the low-level motion planner enables scalability of the high-level plan synthesis with respect to dynamic obstacles, and the high-level mission planner enforces correctness of the low-level motion. We provide formal guarantees for our approach and demonstrate it via physical experiments with ground robots and simulations with a team of quadrotors.

This paper presents an approach to mapping a region of interest using observations from a robotic swarm without localization. The robots have local sensing capabilities and no communication, and they exhibit stochasticity in their motion. We model the swarm population dynamics with a set of advection-diffusion-reaction partial differential equations (PDEs). The map of the environment is incorporated into this model using a spatially-dependent indicator function that marks the presence or absence of the region of interest throughout the domain. To estimate this indicator function, we define it as the solution of an optimization problem in which we minimize an objective functional that is based on temporal robot data. The optimization is performed numerically offline using a standard gradient descent algorithm. Simulations show that our approach can produce fairly accurate estimates of the positions and geometries of different types of regions in an unknown environment.

We present a centralized algorithmic framework for solving multi-robot path planning problems in general, two-dimensional, continuous environments while minimizing globally the task completion time. The framework obtains high levels of effectiveness through the composition of an optimal discretization of the continuous environment and the subsequent fast, near-optimal resolution of the resulting discrete planning problem. This principled approach achieves orders of magnitudes better performance with respect to both speed and the supported robot density. For a wide variety of environments, our method is shown to compute globally near-optimal solutions for 50 robots in seconds with robots packed close to each other. In the extreme, the method can consistently solve problems with hundreds of robots that occupy over 30% of the free space.

Target tracking is a fundamental problem in robotics research and has been the subject of detailed studies over the years. In this paper, we introduce a new formulation, based on the mathematical concept of random finite sets, that allows for tracking an unknown and dynamic number of mobile targets with a team of robots. We show how to employ the Probability Hypothesis Density filter to simultaneously estimate the number of targets and their positions. Next, we present a greedy algorithm for assigning trajectories to the robots to actively track the targets. We prove that the greedy algorithm is a 2-approximation for maximizing submodular tracking objective functions. We examine two such functions: the mutual information between the estimated target positions and future measurements from the robots, and the expected number of targets detected by the robot team. We provide extensive simulation evaluations using a real-world dataset.