Experimental Robotics

This paper presents a preliminary investigation on a one-degree-of-freedom programmable inertia generator. An inertia generator is a hand-held haptic device that has a programmable inertia. By moving internal masses in reaction to accelerations induced by the user, the effective intertia of the device is modified in order to render a prescribed perceived inertia. In this paper, a one-degree-of-freedom device with one internal moving mass is proposed. The dynamic modelling of the system is first presented. Then, a controller is designed to produce the appropriate motion of the internal mass in reaction to the acceleration induced by the user. A prototype is presented and experimental results are discussed.

In this paper, we propose a novel design of end-effectors that is specialized in caging manipulation. Caging manipulation has several advantages comparing with traditional grasping manipulation. For example, caging can allow small gap/margin between end-effectors and a target object, making the manipulator relieved from constant contact and precise control. Therefore, caging manipulator can avoid many problems from dynamics. Regardless of its advantages, intelligent caging manipulators have not be realized. This is because, for one thing, it may demand many actuators to realize flexible geometrical constraint (caging), for the other thing, kinematic constraints of a general purpose manipulator prevents us from applying direct caging approaches.We address this problem by introducing a novel design/framework of end-effectors that is inspired by ROBOTWORLD. The framework utilizes permanent magnet inductive traction method. The method is suitable for coexistence of multiple robots and for reduction of actuator number by sharing the same actuators.We discuss the concept and the basic framework of the proposed caging manipulator and development of a finger component prototype.After that we conduct basic experiments to evaluate the feasibility of caging manipulation and to reveal the obstacles (challenges) for our manipulator.

This paper shows conceptually and experimentally that underactuated robotic hands can stably grasp and manipulate objects placed between the fingertips. Small objects grasped between a planar pair of two-link underactuated fingers having one tendon each are shown to be manipulable over a range of in-hand configurations that can be predicted analytically. The manifold of predicted stable configurations is found by seeking the minimum energy configuration of the elastic fingers under constraints from the actuator tendons and the contact constraints with the grasped objects. Experimental results are shown from HANDLE, a novel underactuated hand capable of a variety of dexterous in-hand tasks.

This paper details the design, fabrication and experimental verification of a complete, tetherless, pressure-operated soft robotic platform. Miniature CO

2

cartridges in conjunction with a custom pressure regulating system are used as an onboard pressure source and embeddable electro-permanent magnet (EPM) [9] valves [13] are used to address supporting hardware requirements. It is shown that this system can repeatedly generate and regulate supply pressure while driving a fluidic elastomer actuator (FEA) [7, 14, 13]. To demonstrate our approach in creating tetherless soft mobile robots, this paper focuses on an example case-study: a soft robotic fish. An underactuated propulsion system emulating natural caudal fin and peduncle movement is designed, fabricated, and subsequently experimentally characterized.

Piezoelectric stick-slip actuators have become the foundation of modern micromanipulation. Due to difficulty in closed-loop control with manipulators that use piezoelectric stick-slip actuators, methods for open-loop control with a human in the loop have been developed. The utility of such methods depends directly on the accuracy of the open-loop models of the manipulator. Prior research has shown that modeling of piezoelectric actuators is not a trivial task as they are known to suffer from nonlinearities that degrade their performance. In this paper, we study the effect of static (non-inertial) loads on a prismatic and a rotary piezoelectric stick-slip actuator, and obtain a model relating the step size of the actuator to the load. The actuator-specific parameters of the model are calibrated by taking measurements in specific configurations of the manipulator. Results comparing the obtained model to experimental data are presented.

Physical human-robot interaction (pHRI) is a highly challenging research topic: it requires real-time decision making capabilities by the robot; it involves the human as a source of uncertainty in the coupled dynamical system; and the quality of interaction cannot be evaluated by classical objective measures only but requires psychological experiments. Here we propose a rapid prototyping system in order to develop and evaluate methods for planning, learning, and control enabling

pro-active and goal-directed physical robotic assistance

to the human. With this rapid prototyping system we are able to quantify the benefits of two novel methods that combine feedback planning and learning from demonstration in a cooperative load-transport task.

To approach robustness and optimal performance, biological musculoskeletal systems can adapt their impedance while interacting with their environment. This property has motivated modern robotic designs including variable-impedance actuators and control methods, based on the capability to vary visco-elastic properties actively or passively. Even though variable-impedance actuation and impedance control in robotics is resolved to a great part, a general set of rules by which impedance is adjusted related to the task at hand is still lacking. This paper aims to fill this gap by providing a method to estimate the stiffness of the human arm in more than two degrees of freedom by perturbation. To overcome ill-conditionedness of the impedance and inertial matrices, we propose and validate methods to separately identify inertial and stiffness parameters. Finally, a model is proposed to estimate the joint stiffness from EMG-measurements of muscle activities.

Human-centered robotics draws growing interest in utilizing pneumatic artificial muscles (PAMs) for robots to cooperate with humans. In order to address the limited control performance which prevents PAMs from being more widely used, a hybrid actuation scheme has been proposed to combine PAMs and a low inertia DC motor, and presented significantly improved control performance without loss of robot safety.While the DC motor provides high precision and reliability, the small motor has, however, difficulties in dealing with the large stored energies of the PAMs, especially in the events of PAMs failure and large initial load changes. In order to further ensure robot safety, we developed a new hybrid actuation scheme with PAMs (macro) and a particle brake (mini), which provides high torque-toweight ratio and inherent stability.We then conducted comparative studies between hybrid actuations with (1) a DC motor and (2) a brake in terms of robot safety and performance. Experimental comparisons show that the hybrid actuation with PAMs and a brake provides higher energy efficiency for control bandwidths under 2 Hz, and is capable of effectively reducing large impacts due to the brake’s high torque capacity and passive energy dissipation. These comparative studies provide insight that the hybrid actuation with PAMs and a brake can be a competitive solution for the applications that require high efficiency, but accept a relatively low control performance, for example, a waist joint.

Recent investigations into biological locomotion have resulted in the development of reduced order templates that emphasize the role of lateral dynamics in achieving rapid and robust fore-aft movement, such as the Full-Goldman model for dynamic climbing and the Lateral Leg Spring model for horizontal plane running. The observation of individual animals demonstrating locomotion via both of these models motivates the development of a single platform that can do so as well. However, a drawback in developing a robot directly from these models stems from both having a bipedal configuration. While a bipedal robot could be designed, the restriction of control approaches, reduction in stability, and preclusion of leg differentiation motivates the development of a platform with additional limbs. In this study, we describe the development of the first quadrupedal platform capable of instantiating the Full-Goldman model, as well as the Lateral Leg Spring model. In particular, the climbing behavior is characterized and the effect of rear leg posture is examined for locomotion on a vertical surface.We demonstrate that climbing behavior can be impacted by the configuration of the rear legs and that minimizing the magnitude of rear leg sprawl may improve efficiency, while rear sprawl postures with a larger magnitude may improve robustness.

In the future robots will have to operate autonomously for long periods of time. To achieve this, they need to be able to learn directly from their environment without human supervision. The use of clustering methods is one possibility to tackle this challenge. Here we present extensions to affinity propagation, a clustering algorithm proposed by Frey and Dueck [5], which makes it suitable for real-time and long-term use in robotics applications. The proposed extension, called meta-point affinity propagation, introduces so called meta-points which increase the performance of the clustering and allows for incremental usage. Additionally we propose a method that enables us to obtain probabilistic cluster assignments from any affinity propagation based clustering method.We show experimental results on the quality and speed of meta-point affinity propagation as well as the probabilistic cluster assignments. Furthermore, we demonstrate how meta-point affinity propagation allows us to process data sets much larger then what affinity propagation is able to handle.

We consider the assembly of a three dimensional (3D) structure by a team of heterogeneous robots capable of online sensing and error correction during the assembly process. The automated assembly problem is posed as a general 3D tiling problem where the assembly components/tiles consist of various shapes and sizes. For a desired 3D structure, we first compute the partition of the assembly strategy into

N

c

sub-components that can be executed in parallel by a team of

N

c

assembly robots. To enable online error detection and correction during the assembly process, mobile robots equipped with visual depth sensors are tasked to scan, identify, and track the state of the structure. The objective is to enable online detection of missing assembly components and reassignment of these components to the team of assembly robots. We present the development of the planning, sensing, and control strategies employed and report on the experimental validation of these strategies using our multi-robot testbed.

Real-world sword-fighting between human opponents requires extreme agility, fast reaction time, and dynamic perception capabilities. In this paper, we present experimental results achieved with a 3D vision system and a highly reactive control architecture which allows a robot to sword fight against human opponents. An online trajectory generator is used as an intermediate layer between low-level trajectory-following controllers and high-level visual perception. This architecture allows robots to react nearly instantaneously to the unpredictable human motions perceived by the vision system as well as to sudden sword contacts detected by force and torque sensors. Results show how smooth and highly dynamic motions are generated on-the-fly while using the vision and force/torque sensor signals in the feedback loops of the robot motion controller.

In this paper, we present two unique aerobots: the spherical blimp used for satellite emulation and the cubic blimp developed for use in floating architecture and visual art. The blimp designs bear a number of similarities, in particular, their construction with an exoskeleton, full actuation to enable six-dof motion and requirement for autonomous localization. Experimental results are presented to demonstrate the closed-loop control for station-keeping, as well as the selected performance statistics such as maximum speeds attained and time the aerobots can remain afloat. Additional qualitative results are presented from the experiments with satellite capture and artistic performances and common challenges with further use in the intended and new applications will be outlined.

The main contribution of this paper is the experimental validation of a decentralized Receding Horizon Mixed Integer Nonlinear Programming (RH-MINLP) framework that can be used to solve the Multi-Vehicle Path Coordination (MVPC) problem. The MVPC problem features path-constrained vehicles that begin their transit from a fixed starting point and move towards a goal point along fixed paths so as to avoid collisions with other robots and static obstacles. This framework allows to solve for time optimal velocity profiles for such robots in the presence of constraints on kinematics, dynamics, collision avoidance, and inter-robot communication connectivity. Experiments involving up to five (5) robots operating in a reasonably complex workspace are reported. Results demonstrate the effect of communication connectivity requirements on robot velocity profiles and the effect of sensing and actuation noise on the path-following performance of the robots. Typically, the optimization improved connectivity at no appreciable cost in journey time, as measured by the time of arrival of the last-arriving robot.

This paper presents a trajectory planning method enabling autonomous robots to approach people in dynamic environments to initiate a conversation proactively. It is shown how integrating human inspired parameters in optimal control based motion planning enables people to predict and read the purpose of a motion more easily.

Experimental evaluations in literature propose to incorporate human-like aspects since the intended action becomes more comprehensible for humans. Therefore, factors like approach speed, distance to the person, positioning near the person, trajectory shape, and the avoidance method are adopted from human behavior to generate motions. The presented trajectory planner is designed to improve the human-like appearance of an approach motion implementing these aspects. Human-likeness is evaluated according to naturalness and comfort of the approach behavior. By executing corresponding trajectories, the approach movement appears more natural and the intended action is easier to predict for humans.

This paper formulates the motion planning procedure as an optimal control problem. Human-like behavior is generated through specific constraints and cost. In order to achieve correct timing, appropriate trajectory shape and the desired behavior for collision avoidance in a dynamic environment, the optimization is split into three consecutive steps. An implementation of a planning algorithm for dynamic environments, capable of online replanning, is proposed. Experiments conducted with this system showed the appropriateness of speed and distance parameters. Further statistical results confirmed that the shape of a trajectory significantly affects the naturalness of an approach motion.

In this paper, we present our approach to design a brain-computer interface (BCI) that allows the user to perform multitask humanoid control. We efficiently integrate techniques from computer vision and the task-function based control together with the brain-computer interface into an immersive and intuitive control application despite the well-known shortcomings of BCI. This approach is assessed in a user experiment involving 4 subjects who successfully controlled the HRP-2 humanoid robot in a scenario involving both grasping tasks and steering. The user experiences and the interface performances are presented and give a rich insight into future research that can be made to improve and extend such interface.

Situational awareness in rescue operations can be provided by teams of autonomous mobile robots. Human operators are required to teleoperate the current generation of mobile robots for this application; however, teleoperation is increasingly difficult as the number of robots is expanded. As the number of robots is increased, each robot may interfere with one another and eventually decrease mapping performance. Through careful consideration of robot team coordination and exploration strategy, large numbers of mobile robots be allocated to accomplish the mapping task more quickly and accurately.

Despite years of research in the area of robotics, the vast majority of industrial robots are still used in “teach-repeat” mode. This requires that the workpiece be in exactly the same position and orientation every time. In many high-volume robotics applications, this is not a problem, since the parts are likely to be fixtured anyway. However, in small to medium lot applications, this can be a significant limitation. The motivation for this project was a corporation who wanted to explore the use of visual control of a manipulator to allow for automated teaching of robot tasks for parts that are run in small lot sizes.

We investigate the reconstruction of an environmental scalar field using robotic mobility and heterogeneous sensing. Using two land-based, immobile, co-located spectrally selective cameras, and a non-contact infraredbased temperature sensor on a quadcopter, we study the problem of reconstructing the surface temperature of the ground under survey. Both land units — a thermographic camera for low-resolution thermal images and a commercial digital camera for high resolution truecolor images — are mounted on an elevated camera rig. We explore methods for field reconstruction using a combination of the three imaging sensors. First, we show that the quadcopter data is correlated with the synoptic snapshots obtained by the thermal imaging camera. Next, we demonstrate upsampling of the low-resolution thermal camera data with truecolor images. This results in high-resolution reconstruction of the temperature field. Finally, we discuss adaptive sampling techniques that utilize the mobility of the quadcopter to ‘fill the gaps’ in data acquired by the thermal imaging camera. Our work experimentally demonstrates the feasibility of heterogeneous sensing and mobility to effectively reconstruct environmental fields.

In this work, we detail a methodology for estimating the state of a microaerial vehicle (MAV) as it transitions between different operating environments with varying applicable sensors. We ensure that the estimate is smooth and continuous throughout and provide an associated quality measure of the state estimate. We address the challenge of maintaining consistency between local and global measurements and propose a strategy to recursively estimate the transform between different coordinate frames. We close with experiments that validate the approach and the resulting performance as a MAV navigates between mixed indoor and outdoor environments.

The dynamic response and performance of a micro UAV is greatly influenced by its aerodynamics which in turn is affected by the interactions with features in the environment in close proximity. In the paper we address the modeling of quadrotor robots in different flight conditions that include relative wind velocity and proximity to the ground, the ceiling and other robots. We discuss the incorporation of these models into controllers and the use of a swarm of robots to map features in the environment from variations in the aerodynamics.

In this paper, we study estimator inconsistency in Vision-aided Inertial Navigation Systems (VINS). We show that standard (linearized) estimation approaches, such as the Extended Kalman Filter (EKF), can fundamentally alter the system observability properties, in terms of the number and structure of the unobservable directions. This in turn allows the influx of spurious information, leading to inconsistency. To address this issue, we propose an Observability-Constrained VINS (OC-VINS) methodology that explicitly adheres to the observability properties of the true system.We apply our approach to the Multi-State Constraint Kalman Filter (MSC-KF), and provide both simulation and experimental validation of the effectiveness of our method for improving estimator consistency.

Ultra-wideband (UWB) localization is a recent technology that promises to outperform many indoor localization methods currently available. Despite its desirable traits, such as precision and high material penetrability, the resolution of non-line-of-sight (NLOS) signals remains a very hard problem and has a significant impact on the localization accuracy. In this work, we address the peculiarities of UWB error behavior by building models that capture the spatiality as well as the multimodal nature of the error statistics. Our framework utilizes tessellated maps that associate multimodal probabilistic error models to localities in space. In addition to our UWB localization strategy (which provides absolute position estimates), we investigate the effects of collaboration in the form of relative positioning. We test our approach experimentally on a group of ten mobile robots equipped with UWB emitters and extension modules providing inter-robot relative range and bearing measurements.

We develop algorithms for estimation and control that allow a team of robots equipped with range sensors to localize an unknown target in a known but complex environment. We present an experimental model for radio-based time-of-flight range sensors. Adopting a Bayesian approach for estimation, we then develop a control law which maximizes the mutual information between the robot’s measurements and their current belief of the target position. We describe experimental results for a robot team localizing a stationary target in several representative indoor environments in which the unknown target is reliably localized with an error well below the typical error for individual measurements.

In this work, we consider a group of differential-wheeled robots endowed with noisy relative positioning capabilities. We develop a decentralized approach based on a receding horizon controller to generate, in real-time, trajectories that guarantee the convergence of our robots to a common location (i.e. rendezvous).Our receding horizon controller is tailored around two numerical optimization methods: the hybrid-state A* and trust-region algorithms. To validate both methods and test their robustness to computational delays, we perform exhaustive experiments on a team of four real mobile robots equipped with relative positioning hardware.

One of the many challenges in building robust and reliable autonomous systems is the large number of parameters and settings such systems often entail. The traditional approach to this task is simply to have system experts hand tune various parameter settings, and then validate them through simulation, offline playback, and field testing. However, this approach is tedious and time consuming for the expert, and typically produces subpar performance that does not generalize. Machine learning offers a solution to this problem in the form of learning from demonstration. Rather than ask an expert to explicitly encode his own preferences, he must simply demonstrate them, allowing the system to autonomously configure itself accordingly. This work extends this approach to the task of learning driving styles and maneuver preferences for an autonomous vehicle. Head to head experiments in simulation and with a live autonomous system demonstrate that this approach produces better autonomous performance, and with less expert interaction, than traditional hand tuning.

Recently introduced RGB-D cameras are capable of providing high quality synchronized videos of both color and depth. With its advanced sensing capabilities, this technology represents an opportunity to dramatically increase the capabilities of object recognition. It also raises the problem of developing expressive features for the color and depth channels of these sensors. In this paper we introduce hierarchical matching pursuit (HMP) for RGB-D data. HMP uses sparse coding to learn hierarchical feature representations from raw RGB-D data in an unsupervised way. Extensive experiments on various datasets indicate that the features learned with our approach enable superior object recognition results using linear support vector machines.

As robots become more ubiquitous and capable of performing complex tasks, the importance of enabling untrained users to interact with them has increased. In response, unconstrained natural-language interaction with robots has emerged as a significant research area. We discuss the problem of parsing natural language commands to actions and control structures that can be readily implemented in a robot execution system. Our approach learns a parser based on example pairs of English commands and corresponding control language expressions. We evaluate this approach in the context of following route instructions through an indoor environment, and demonstrate that our system can learn to translate English commands into sequences of desired actions, while correctly capturing the semantic intent of statements involving complex control structures. The procedural nature of our formal representation allows a robot to interpret route instructions online while moving through a previously unknown environment.

Grasping an object is usually only an intermediate goal for a robotic manipulator. To finish the task, the robot needs to know where the object is in its hand and what action to execute. This paper presents a general statistical framework to address these problems. Given a novel object, the robot learns a statistical model of grasp state conditioned on sensor values. The robot also builds a statistical model of the requirements of the task in terms of grasp state accuracy. Both of these models are constructed by offline experiments. The online process then grasps objects and chooses actions to maximize likelihood of success. This paper describes the framework in detail, and demonstrates its effectiveness experimentally in placing, dropping, and insertion tasks. To construct statistical models, the robot performed over 8000 grasp trials, and over 1000 trials each of placing, dropping and insertion.

BCI (Brain Computer Interface) technology shows great promise in the field of assistive robotics. In particular, severely impaired individuals lacking the use of their hands and arms would benefit greatly from a robotic grasping system that can be controlled by a simple and intuitive BCI. In this paper we describe an end-to-end robotic grasping system that is controlled by only four classified facial EMG signals resulting in robust and stable grasps. A front end vision system is used to identify and register objects to be grasped against a database of models. Once the model is aligned, it can be used in a real-time grasp planning simulator that is controlled through a non-invasive and inexpensive BCI interface in both discrete and continuous modes. The user can control the approach direction through the BCI interface, and can also assist the planner in choosing the best grasp. Once the grasp is planned, a robotic hand/arm system can execute the grasp. We show results in using this system to pick up a variety of objects in real-time, from a number of different approach directions, using facial BCI signals exclusively. We believe this system is a working prototype for a fully automated assistive grasping system.

Ensuring proper living conditions for an ever growing number of elderly people is a significative challenge for many countries. The difficulty and cost of hiring and training specialized personnel has fostered research in assistive robotics as a viable alternative. In this context, an ideally suited and very relevant application is to transport people with reduced mobility. This may involve either autonomous or semi-autonomous transportation devices such as cars and wheelchairs.

For a working solution, a number of problems including safety, usability and economic feasibility have to be solved. This paper presents PAL’s robotic wheelchair, an experimental platform to study and provide solutions to many of the aforementioned problems.

We present the design, implementation, and user study evaluation of a socially assistive robot (SAR) system designed to engage elderly users in physical exercise aimed at achieving health benefits and improving quality of life. We discuss our design methodology, which incorporates insights from psychology research in the area of intrinsic motivation, and focuses on maintaining engagement through personalized social interaction. We describe two user studies conducted to test our design principles in practice with our system. The first study investigated the role of praise and relational discourse in the exercise system by comparing a relational robot coach to a non-relational robot coach. The second study compared physical vs. virtual embodiment in the task scenario. The results of both studies demonstrate the feasibility and overall effectiveness of the robot exercise system.

The creation of a robot chef represents a grand challenge for the field of robotics. Cooking is one of the most important activities that takes place in the home, and a robotic chef capable of following arbitrary recipes would have many applications in both household and industrial environments. The kitchen environment is a semi-structured proving ground for algorithms in robotics. It provides many computational challenges, such as accurately perceiving ingredients in cluttered environments, manipulating objects, and engaging in complex activities such as mixing and chopping.

As humans we use our arms and bodies in addition to our hands to grasp objects. We (and robots) often cannot use caging or closure strategies when manipulating bulky objects. This paper studies manipulating such objects in the context of a particular task: equalizing a load across the arms of a two-armed robot. Our PR2 robot performs this task using only proprioceptive force sensing and a simple, reactive equalization strategy.We demonstrate the robot robustly performing this task using numerous and various objects (

e.g.

, boxes, pipes, broomsticks, backpacks).

This work aims at defining a suitable postural synergies subspace for the DEXMART Hand from observation of human hand grasping postures. Previous works were carried out on a preliminary prototype (the UB Hand IV), without neither proprioceptive integrated sensors nor external sensors, by means of a joint-to- joint mapping technique. Using an RGB camera and depth sensor for 3D motion capture, the human hand palm pose and fingertip positions have been measured for a reference set of grasping postures. The proposed method for the determination of the synergies subspace is based on the kinematics mapping from the human hand to the robotic hand using data from experiments involving five subjects. The subjects’ hand configurations have been mapped to the robotic hand by matching the hand pose and fingertip positions and applying a closed-loop inverse kinematic algorithm. Suitable scaling factors have been used to adapt the DEXMART Hand kinematics to the subjects’ hand dimension. By means of Principal Component Analysis (PCA), the kinematic patterns of the first three predominant synergies have been computed and a brief comparison with the previous method and kinematics is reported. Finally, a synergy-based control strategy has been used for testing the efficiency of the grasp synthesis method.

We present DARRT, a sampling-based algorithm for planning with multiple types of manipulation. Given a robot, a set of movable objects, and a set of actions for manipulating the objects, DARRT returns a sequence of manipulation actions that move the robot and objects from an initial configuration to a final configuration. The manipulation actions may be non-prehensile,meaning that the object is not rigidly attached to the robot, such as push, tilt, or pull. We describe a simple extension to the RRT algorithm to search the combined space of robot and objects and present an implementation of DARRT on the Willow Garage PR2 robot.

Motion planning for robotic manipulation of clothing is a challenging problem as clothing articles have high-dimensional configuration spaces and are computationally expensive to simulate. We present an algorithm for robotic cloth folding that, given a sequence of desired folds, outputs a motion plan consisting of a sequence of primitives for a robot to fold the cloth. Previous work on cloth folding does not take into account the constraints of the robot, and thus produces plans which are often infeasible given the kinematics of robots like the Willow Garage PR2. In this paper we introduce a class of motion primitives that start and end in a subset of the cloth’s state space. To find a sequence of primitives that achieves all required folds, the algorithm takes advantage of the partial ordering inherent in folding, and produces a time-optimal motion plan (given the set of primitives) for the robot if one exists. We describe experiments with a general purpose mobile robotic platform, the PR2, folding articles that require dragging and base motion in addition to folding. Our experiments show that (1) many articles of clothing conform well enough to the assumptions made in our model and (2) this approach allows our robot to perform a wide variety of folds on articles of various sizes and shapes.

Magnetic Resonance Imaging (MRI) provides superior soft-tissue contrast. But the strong magnetic field inside theMRI bore and the limited scanner bore size restricts direct means of breast biopsy under real-time imaging. Current blind targeting approach based on MR images obtained a priori sometimes requires multiple needle insertions if the tool tip position is compromised. A MRI-compatible robot that can be teleoperated is thus desired to take advantage of the real-time MR imaging and avoid multiple needle insertions. In this paper, we present our initial work on the development of a master-slave surgical system. The MRI-compatible slave robot is actuated with five pneumatic cylinders and one piezo motor and operates inside the MRI bore. The master robot provides an intuitive manipulation platform for the physician. The MRI experiment shows that the slave robot does not induce visually-detectable distortion in the MR images and can be safely operated inside the MRI.

Cardiac catheters allow physicians to access the inside of the heart and perform therapeutic interventions without stopping the heart or opening the chest. However, conventional manual and actuated cardiac catheters are currently unable to precisely track and manipulate the intracardiac tissue structures because of the fast tissue motion and potential for applying damaging forces. This paper addresses these challenges by proposing and implementing a robotic catheter system that use 3D ultrasound image guidance and force control to enable constant contact with a moving target surface in order to perform an interventional procedure, in this case tissue ablation. The robotic catheter system, consisting of a catheter module, ablation and force sensing end effector, drive system, and image-guidance and control system, was commanded to apply a constant force against a moving target using a position-modulated force control method. As compared to a manual catheter system, the robotic catheter was able to apply a more consistent force on the target while maintaining ablation electrode contact with 97% less RMS contact resistance variation. These results demonstrate that the 3D ultrasound guidance and force control allow the robotic system to maintain better contact with a moving tissue structure, thus allowing for more accurate and repeatable tissue ablation procedures.

This paper presents a system for mechanically characterizing single cells using automated micropipette aspiration. Using vision-based control and position control, the system controls a micromanipulator, a motorized translation stage, and a custom-built pressure system to position a micropipette (4

μ

m opening) to approach a cell, form a seal, and aspirate the cell into the micropipette for quantifying the cell’s elastic and viscoelastic parameters as well as viscosity. Image processing algorithms were developed to provide controllers with real-time visual feedback and to accurately measure cell deformation behavior on line. Experiments on both solid-like and liquid-like cells demonstrated that the system is capable of efficiently performing single-cell micropipette aspiration and has low operator skill requirements.

Sperm selection plays a significant role in in

vitro

fertilization (IVF). Approaches for assessing sperm quality include non-invasive techniques based on sperm morphology and motility as well as invasive techniques for checking DNA integrity. In 2006, a new device using hyaluronic acid (HA) coated dish for sperm selection was cleared by the Food and Drug Administration (FDA) and entered IVF clinics. In this technique, only sperms with DNA integrity bind to the HA droplet, after which these bound sperm stop revealing head motion and their tail movement becomes more vigorous. However, selecting a single sperm cell from among HAbound sperms is

ad hoc

in IVF clinics. Different from existing sperm tracking algorithms that are largely limited to tracking sperm head only and are only able to track one sperm at a time, this paper presents a multi-sperm tracking algorithm that tracks both sperm heads and low-contrast sperm tails. The tracking results confirm a significant correlation between sperm head velocity and tail beating amplitude; demonstrate that sperms bound to HA generally have a higher velocity (before binding) than those sperms that are not able to bind to HA microdots; and quantitatively reveal that HA-bound sperms’ tail beating amplitudes are different among HA-bound sperms.

This work addresses the characterization and control of Magnetotactic Bacterium (MTB) which can be considered as a biological microrobot. Magnetic dipole moment of the MTB and response to a field-with-alternating-direction are characterized. First, the magnetic dipole moment is characterized using four techniques, i.e., Transmission Electron Microscopy images,

flip-time, rotating-field

and

u-turn

techniques. This characterization results in an average magnetic dipole moment of 3.32x10

16

A.m2 and 3.72x10

16

A.m

2

for non-motile and motile MTB, respectively. Second, the frequency response analysis of MTB shows that its velocity decreases by 38% for a field-with-alternating-direction of 30 rad/s. Based on the characterized magnetic dipole moment, the magnetic force produced by our magnetic system is five orders-of-magnitude less than the propulsion force generated by the flagellum of the MTB. Therefore,

point-to-point

positioning of MTB cannot be achieved by exerting a magnetic force. A closed-loop control strategy is devised based on calculating the position tracking error, and capitalizes on the frequency response analysis of the MTB.

Point-to-point

closed-loop control of MTB is achieved for a reference set-point of 60

μ

m with average velocity of 20

μ

m/s. The closed-loop control system positions the MTB within a region-of-convergence of 10

μ

m diameter.

Robots are used in different applications to enhance human performance and in the future, these interactions will become more frequent. In order to achieve this human augmentation, the cooperation must be very intuitive to the human operator. This paper proposes a computed-torque control scheme for pHRI using admittance control. The admittance model is first introduced. Then, the robot identification, the computed-torque approach and the saturation considerations are addressed. The intelligent assist device used for the experiments is then presented. Finally, experimental results that demonstrate the performance of the algorithm are provided.

This paper presents a methodology for computationally efficient, direct trajectory generation using sampling with the minimum time cost function, where only the initial and final positions and velocities of the trajectory are specified. The approach is based on a randomized A* algorithm called Sampling-Based Model Predictive Optimization (SBMPO) that exclusively samples in the input space and integrates a dynamic model of the system. The paper introduces an extended kinematic model, consisting of the standard kinematic model preceded by two integrators. This model is mathematically a dynamic model and enables SBMPO to sample the acceleration and provide the acceleration, velocity, and position as functions of time that are needed by a typical trajectory tracking controller. A primary contribution of this paper is the development of an appropriate “optimistic A* heuristic” (i.e, a rigorous lower bound on the chosen cost) based on the solution of a minimum time control problem for the system q̈ = u; this heuristic is a key enabler to fast computation of trajectories that end in zero velocity. Another contribution of this paper is the use of the extended kinematic model to develop a trajectory generation methodology that takes into account torque constraints associated with the regular dynamic model without having to integrate this more complex model as has been done previously. This development uses the known form of the trajectory following control law. The results are initially illustrated experimentally using a 1 degree of freedom (DOF) manipulator lifting heavy loads, which necessitates the development of trajectories with appropriate momentum characteristics. Further simulation results are for a 2 DOF manipulator.

This paper presents a mechanical system that fundamentally mimics a human musculo-skeletal system aiming for using it in anthropomorphic robots or artificial limbs for disabled persons. At first, it introduces a mechanical module called ANLES (Actuator with Non-Linear Elasticity System). A new type of ANLES; rotary-type ANLES is introduced first in this paper, in addition to the formerly developed ANLES; linear type ANLES. They can be used like a voluntary muscle in a musculo-skeletal structure. Next it derives dual equations to independently control joint angle and joint stiffness, in which antagonistic alignment of the ANLESes similar to a musclo-skeletal system is premised. It follows to show an application of the two types of ANLES into a three DOF artificial joint arranged to use as a wrist joint of an anthropomorphic robot. The experimental results of the joint stiffness and joint angle control elucidates that the developed mechanism effectively mimics the human musculo-skeletal system.

In digital computer based controllers, efficient sampling mechanisms for sensors as well as controllers is of great importance. In this paper, we are interested in designing controllers that result in low average frequency of control updates while simultaneously ensuring stability of the robotic system. We experimentally investigate a non-periodic state-triggered control sampling scheme (designed through Lyapunov like analysis) for adaptive tracking controllers in robot manipulators. We implement this scheme on two well known continuous-time adaptive controllers for tracking in robot manipulators and compare their performance heuristically based on the results of experiments performed on a two link planar manipulator.

This work combines semantic maps with hybrid control models, generating a direct link between action and environment models to produce a control policy for mobile manipulation in unstructured environments. First, we generate a semantic map for our environment and design a base model of robot action. Then, we combine this map and action model using the Motion Grammar Calculus to produce a combined robot-environment model. Using this combined model, we apply supervisory control to produce a policy for the manipulation task. We demonstrate this approach on a Segway RMP-200 mobile platform.

We explore the problem of energy-efficient, time-constrained path planning of a solar powered robot embedded in a terrestrial environment. Because of the effects of changing weather conditions, as well as sensing concerns in complex environments, a new method for solar power prediction is desired. We present a method that uses Gaussian Process regression to build a solar map in a data-driven fashion. With this map, we perform energy-optimal path planning using a dynamic programming algorithm. We validate our map construction and path planning algorithms with outdoor experiments, and perform simulations on our solar maps to determine under which conditions the weight of added solar panels is worthwhile for a mobile robot.

We present a method for determining 3-dimensional, local ground displacements caused by an earthquake. The technique requires pre- and post-earthquake point cloud datasets, such as those collected using airborne Light Detection and Ranging (Lidar). This problem is formulated as a point cloud registration problem in which the full point cloud is divided into smaller windows, for which the local displacement that best restores the post-earthquake point cloud onto its pre-earthquake equivalent must be found. We investigate how to identify the size of window to be considered for registration. We then present an information theoretic approach that classifies whether a region contains an earthquake fault. These methods are first validated on simulated earthquake datasets, for which the input displacement field is known, and then tested on a real earthquake. We show results and error analyses for a variety of different window sizes, as well as results for our fault detection algorithm.

Crop yield estimation is an important task in apple orchard management. The current manual sampling-based yield estimation is time-consuming, labor-intensive and inaccurate. To deal with this challenge, we developed a computer vision-based system for automated, rapid and accurate yield estimation. The system uses a two-camera stereo rig for image acquisition. It works at nighttime with controlled artificial lighting to reduce the variance of natural illumination. An autonomous orchard vehicle is used as the support platform for automated data collection. The system scans both sides of each tree row in orchards. A computer vision algorithm detects and registers apples from acquired sequential images, and then generates apple counts as crop yield estimation. We deployed the yield estimation system in Washington state in September, 2011. The results show that the system works well with both red and green apples in the tall-spindle planting system. The crop yield estimation errors are -3.2% for a red apple block with about 480 trees, and 1.2% for a green apple block with about 670 trees.

Several important forms of robotic environmental monitoring involve estimating a spatial field from comparatively few measurements. A number of researchers use linear least squares estimation techniques, frequently either the geostatistical Kriging framework or a Gaussian Process regression formulation, that provide estimates of quantities of interest at unmeasured locations. These methods enable selection of sample locations (e.g., for adaptive sampling) by quantifying uncertainty across the scalar field. This paper assesses the role of pose uncertainty and measurement error on variance of the estimated spatial field. We do this through a systematic empirical comparison of scalar fields reconstructed from measurements taken with our robot using multiple imperfect sensors and actively estimating its pose. We implement and compare two models of variance: Kriging Variance (KV) and Interpolation Variance (IV), illustrating that the latter—which has not been used in a robotics context before—has several advantages when used for online planning of sampling tasks. Using two separate experimental scenarios, we assess the estimated variance in scalar fields constructed from measurements taken by robots. Physical robots sampling within our office building suggest that using IV to select sampling sites gathers more data for a given time window (45% more than KV), travels a shorter distance to collect the same number of samples (25% less than KV), and has a promising speed-up with multiple robots. Water quality data from an Autonomous Underwater Vehicle survey of Lake Pleasant, AZ. also show that IV produces better qualities for given a distance and time.

This work is motivated by the desire to covertly track mobile targets, either animal or human, in previously unmapped outdoor natural environments using off-road robotic platforms with a non-negligible acoustic signature. The use of robots for stealthy surveillance is not new. Many studies exist but only consider the navigation problem to maintain visual covertness. However, robotic systems also have a significant acoustic footprint from the onboard sensors, motors, computers and cooling systems, and also from the wheels interacting with the terrain during motion. All these can jepordise any visual covertness. In this work, we experimentally explore the concepts of opportunistically utilizing naturally occurring sounds within outdoor environments to mask the motion of a robot, and being visually covert whilst maintaining constant observation of the target. Our experiments in a constrained outdoor built environment demonstrate the effectiveness of the concept by showing a reduced acoustic signature as perceived by a mobile target allowing the robot to covertly navigate to opportunistic vantage points for observation.

Exploration of underwater environments, such as coral reefs and ship wrecks, is a difficult and potentially dangerous tasks for humans, which naturally makes the use of an autonomous robotic system very appealing. This paper presents such an autonomous system, and shows its use in a series of experiments to collect image data in an underwater marine environment.We presents novel contributions on three fronts. First, we present an online topic-modeling based technique to describe what is being observed using a low dimensional semantic descriptor.

We examine the problem of planning dives for an Autonomous Underwater Vehicle (AUV) to generate a dense bathymetric map using sidescan sonar. The three key challenges in this scenario are (1) proper modeling of the local uncertainty of the 3D reconstruction, (2) efficient dive planning to reduce this uncertainty, and (3) determination of when to re-plan adaptively based on new information. To address these challenges, we propose using non-parametric Bayesian regression to model the expected accuracy of the map, which provides principled cost functions for planning subsequent dives. In addition, we propose an efficient greedy method to reduce this uncertainty, and we show that it achieves theoretically bounded performance given assumptions on the sensor model and the form of the uncertainty function. We present experiments on the propeller-driven YSI EcoMapper AUV equipped with a sidescan sonar in an inland lake. The experiments demonstrate the benefit of efficient dive planning, with our results providing performance gains of up to 83% versus standard lawnmower patterns.

In the coastal ocean, biological and physical dynamics vary on spatiotemporal scales spanning many orders of magnitude. At large spatial (

O

(100km)) and temporal (

O

(weeks to months)) scales, traditional shipboard and moored measurements are very effective at quantifying mean and varying oceanic properties. At scales smaller than the internal Rossby radius (

O

(10km) for typical coastal stratification at mid-latitude), horizontal, vertical and temporal inhomogeneity is the rule rather than the exception.

We present a fully autonomous data muling system consisting of hardware and algorithms. The system allows a robot to autonomously find a sensor node and use high bandwidth, short range optical communication to download 1.2 MB of data from the sensor node and then transport the data back to a base station. The hardware of the system consists of an autonomous underwater vehicle (AUV) paired with an underwater sensor node. The robot and the sensor node use two modes of communication - acoustic for long-range communication and optical for high bandwidth communication. No positioning system is required. Acoustic ranging is used between the sensor node and the AUV. The AUV uses the ranging information to find the sensor node by means of either stochastic gradient descent, or a particle filter. Once it comes close enough to the sensor node where it can use the optical channel it switches to position keeping by means of stochastic gradient descent on the signal quality of the optical link. During this time the optical link is used to download data. Fountain codes are used for data transfer to maximize throughput while minimizing protocol requirements. The system is evaluated in three separate experiments using our Autonomous Modular Optical Underwater Robot (AMOUR), a PANDA sensor node, the UNET acoustic modem, and the AquaOptical modem. In the first experiment AMOUR uses acoustic gradient descent to find the PANDA node starting from a distance of at least 25 m and then switches to optical position keeping during which it downloads a 1.2 MB large file. This experiment is completed 10 times successfully. In the second experiment AMOUR is manually steered above the PANDA node and then autonomously maintains position using the quality of the optical link as a measurement. This experiment is performed two times for 10 minutes. The final experiment does not make use of the optical modems and evaluate the performance of the particle filter in finding the PANDA node. This experiment is performed 5 times successfully.

In this paper, we study a problem encountered during our ongoing efforts to locate radio-tagged fish aggregations with robots. The problem lies at the intersection of search-based methods whose objective is to detect a target, and active target localization methods whose objective is to precisely localize a target given its initial estimate. Real-world sensing constraints such as limited and unknown range, large measurement time, and ambiguity in bearing measurements make it imperative to have an intermediate initialization phase to transition from search to localization.We present a local search strategy aimed at reliably initializing an estimate for a single target based on observations from field experiments.We then extend this strategy to initialize multiple targets, exploiting the proximity of nearby aggregated tagged fish to decrease the cost of initialization per target. We evaluate the performance of our algorithm through simulations and demonstrate its utility through a field experiment where the robot successfully detects, initializes and then localizes nearby targets.

Odometry is an important input to robot navigation systems, and we are interested in the performance of vision-only techniques. In this paper we experimentally evaluate and compare the performance of wheel odometry, monocular feature-based visual odometry, monocular patch-based visual odometry, and a technique that fuses wheel odometry and visual odometry, on a mobile robot operating in a typical indoor environment.

Traversability maps are a global spatial representation of the relative difficulty in driving through a local region. These maps support simple optimisation of robot paths and have been very popular in path planning techniques. Despite the popularity of these maps, the methods for generating global traversability maps have been limited to using a-priori information. This paper explores the construction of large scale traversability maps for a vehicle performing a repeated activity in a boundedworking environment, such as a repeated delivery task.We evaluate the use of vehicle power consumption, longitudinal slip, lateral slip and vehicle orientation to classify the traversability and incorporate this into a map generated from sparse information.

Mobile scanning, i.e., the practice of mounting laser scanners on moving platforms is an efficient way to acquire accurate and dense 3D point clouds of outdoor environments for urban and regional planning and architecture. The mobile scenario puts high requirements on the accuracy of the calibration of the measurement system, as small calibration inaccuracies lead to large errors in the resulting point cloud. We propose a novel algorithm for the calibration of a mobile scanning system that estimates the calibration parameters for

all

sensor components simultaneously without relying on additional hardware. We evaluate the calibration algorithm on several real world data sets where ground truth is available via an accurate geodetic model.

While a significant body of work has been done on grasping objects, there is little prior work on placing and arranging objects in the environment. In this work, we consider placing multiple objects in complex placing areas, where neither the object nor the placing area may have been seen by the robot before. Specifically, the placements should not only be stable, but should also follow human usage preferences.We present learning and inference algorithms that consider these aspects in placing. In detail, given a set of 3D scenes containing objects, our method, based on Dirichlet process mixture models, samples human poses in each scene and learns how objects relate to those human poses. Then given a new room, our algorithm is able to select meaningful human poses and use them to determine where to place new objects.We evaluate our approach on a variety of scenes in simulation, as well as on robotic experiments.

Hand gesture can express rich information. However, existing hand shape recognition methods have several problems. In order to utilize hand gesture in a home automation, we have focused on “wrist contour”, and have developed a wrist-watch-type device that measures wrist contour using photo reflector arrays. In this paper, we try on two challenges: the first is improvement of the hand shape recognition performance, and the second is making clear the effect of personal difference and finding a key to overcome the difference. We collect wrist contour data from 28 subjects and conduct two kinds of experiments. As for the first challenge, three different feature types are compared. The experimental results extract several important contour statistics and the classification rate itself is also improved by introducing multiple subjects’ data for training. As for the second challenge, we compose a resemblance matrix to evaluate resemblance among subjects. The results indicate that training data selection is important to improve the classification performance, especially when we don’t have time to collect enough training data for a new user.

Although fully autonomous robots continue to advance in ability, all points on the spectrum of cooperative interfaces between man and machine continue to have their place. We have developed a suite of operator assist technologies for a small (1 cubic meter volume) high speed robot that is intended to improve both speed and fidelity of control. These aids include fast stability control loops that run on the robot and graphical user interface enhancements that help the operator cope with lost peripheral vision, unstable video, and latency. After implementing the driving aids, we conducted an experiment where we evaluated the relative value of each from the perspective of their capacity to improve driving performance. Over a one week period, we tested 10 drivers in each of four driving configurations for three repetitions of a difficult test course. The results demonstrate that operators of all skill levels can benefit from the aids and that stabilized video and predictive displays are among the most valuable of the features we added.

A critical component of autonomous driving in urban environment is the vehicle’s ability to interact safely and intelligently with the human drivers and on-road pedestrians. This requires identifying the human intentions in real time based on a limited observation history and reacting accordingly. In the context of pedestrian avoidance, traditional approaches like proximity based reactive avoidance, or taking the most likely behavior of the pedestrian into account, often fail to generate a safe and successful avoidance strategy. This is mainly because they fail to take into account the human intention and the inherent uncertainty resulting in identifying such intentions from direct observations.

This work formulates the on-road pedestrian avoidance problem as an instance of the

Intention-Aware Motion Planning

(IAMP) problem, where the human intention uncertainty is incorporated in a principled manner into the planning framework. Assuming a set of all possible pedestrian intentions in the environment, IAMPs generate a Mixed Observable Markov Decision Process (MOMDP), (a factored variant of

Partially Obervable Markov Decision Process

(POMDP)) with the human intentions being the

unobserved

variables. Solving the resulting MOMDP generates a robust pedestrian avoidance policy. In spite of the criticism of POMDPs to be computationally intractable in general, we show that with proper state factorization and latest sampling based approaches the policy can be executed online on a real vehicle on road. We demonstrate this by running the algorithm on a real pedestrian crossing in the NUS campus successfully handling the intentions for multiple pedestrians, even when they are jaywalking. In this paper, we present results in simulation to show the improved performance of the proposed approach over existing methods. Additionally, we present results validating experimentally the assumptions made in formulating the intention aware pedestrian avoidance problem.

This work presents a preliminary step towards safer and effective autonomous navigation in urban environments by incorporating the intentions of pedestrians and other drivers on the road.

Recognizing objects is a fundamental capability for robotic systems but comparing algorithms on similar testing situations remains a challenge. This makes characterizing the current state-of-the-art difficult and impedes progress on the task. We describe a recently proposed benchmark for robotic object recognition, named the UBC Visual Robot Survey,which is a robot-collected dataset of cluttered kitchen scenes. The dataset contains imagery and range data collected from a dense sampling of viewpoints. Objects have been placed in realistic configurations that result in clutter and occlusion, similar to common home settings. This data and accompanying tools for simulation from real data enable the study of robotic recognition methods. They specifically allow focus on specific concerns in robotics such as spatial evidence integration and active perception.We describe the method used to produce the dataset in detail, a suite of testing protocols and the current state-of-the-art performance on the dataset.