Brain, Body and Machine

Abstract. Unlike the industrial robotics domain where the workspace of machines and humans can be segmented, applications of intelligent machines that work in contact with humans are increasing, which involve e.g. haptic interfaces and teleoperators, cooperative material-handling, power extenders and such high-volume markets as rehabilitation, physical training and entertainment. Force and vision play a fundamental role to increase the autonomy of a robotic system, especially in the presence of humans. Vision provides global information on the surrounding environment to be used for motion planning and obstacle avoidance, while force allows adjusting the robot motion so that the local constraints imposed by the environment are satisfied. In order to avoid dangerous collisions and ensure a safe interaction, suitable control strategies based on force and visual feedback can be used while tracking human motion. This paper surveys such strategies and presents some experimental results in a number of significant case studies.

During the past half-century, the cornerstone of treatment for brain metastases has been whole brain irradiation (WBI). WBI has multiple salutary effects including rapid relief of neurological signs and symptoms as well as enhanced local control. Unfortunately, WBI may also engender side effects including memory deficits and decrements in quality of life. Since memory control is thought to be mediated by the hippocampus, attention has been turned to whole brain radiotherapeutic techniques that allow sparing of the hippocampus. In order to be able to minimize dose deposition within the hippocampus, clinicians must be able to confidently identify that structure. However, manually tracing out the hippocampus for each patient is time consuming and subject to individual bias. To this end, an automated method can be very useful for such a task. In this paper, we present a method for extracting the hippocampus from magnetic resonance imaging (MRI) data. Our method is based on a multi-scale shape representation using statistical learning in conjunction with spherical wavelets for shape representation. Indeed, the hippocampus shape information is statistically learned by the algorithm and is further utilized to extract a hippocampus from the given 3D MR image. Results are shown on data-sets provided by Brigham and Women’s Hospital.

We present a new technique for registering ultrasound and magnetic resonance (MR) images in the context of neurosurgery. It involves generating a pseudo-ultrasound (pseudo-US) from a segmented MR image and uses cross-correlation as the cost function to register with ultrasound. The algorithm’s performance is compared to a state-of-the-art technique that uses a median filtered MR images to register with a Gaussian-blurred ultrasound using normalized mutual information (NMI). The two methods are tested on nine tumor cases, including both high- and low-grade gliomas. The pseudo-US method yielded significantly better alignment average than that obtained by NMI (p = 0.0009).  If one case where NMI failed is excluded, the mean distance obtained by the pseudo-US approach (2.6 mm) is slightly lower than the one obtained by NMI (2.8mm), but not significantly so (p = 0.16).  We conclude that the pseudo-US method is more robust for these cases.

In this paper, we present an improvement to an image integration technique called Virtual Mirroring (VM). The basis for this approach is formed on the notion that humans are familiar with seeing multiple views of a scene through reflections off various reflective surfaces. VM overlays secondary views of a scene onto virtual mirrors that can be seen by a primary view of the scene, thereby providing more views of a given scene to the observer. We refer to the cameras that capture the secondary and primary views as Secondary (SC) and Principal (PC) Cameras respectively. In our previous approach, the camera setup was constrained to a specific relative position between the cameras. Moreover, the virtual mirrors we created did not respect the laws of reflection. In this new approach, we assume an object plane on the scene in order to be able to determine the world position of each SC image point and subsequently, the corresponding points of reflection through a virtual mirror. By using an object plane, the camera placement is not constrained and the virtual mirrors are created using equations that adhere to the laws of reflection.

Measuring the difference between two multivariate Gaussians is central to statistics and machine learning. Traditional measures based on the Bhattacharyya coefficient or the symmetric Kullback-Leibler divergence do not satisfy metric properties necessary for many algorithms. This paper proposes a metric for Gaussian densities. Similar to the Bhattacharyya distance and the symmetric Kullback-Leibler divergence, the proposed metric reduces the difference between two Gaussians to the difference between their parameters. Based on the proposed metric we introduce a symmetric and positive semi-definite kernel between Gaussian densities. We illustrate the benefits of the proposed metric in two settings: (1) a supervised problem, where we learn a low-dimensional projection that maximizes the distance between Gaussians, and (2) an unsupervised problem on spectral clustering where the similarity between samples is measured with our proposed kernel.

This paper considers the problem of estimating the brightness of visual stimuli. A number of physical asymmetries are seen to permit determination of brightness that is invariant to certain manipulations of the sensor responses, such as inversion. In particular, the light-dark range asymmetry is examined and is shown to result, over a certain range, in increased variability of sensor responses as scene brightness increases. Based on this observation we propose that brightness can be measured using variability statistics of conditional distributions of image patch values. We suggest that a process of statistical learning of these conditional distributions underlies the Stevens effect.

This paper presents a novel patient repositioning method from limitedangle tomographic projections. It uses a machine learning strategy. Given a single planning CT image (3D) of a patient, one applies patient-specific training. Using the training results, the planning CT image, and the raw image projections collected at the treatment time, our method yields the difference between the patient’s treatmenttime postition and orientation and the planning-time position and orientation. In the training, one simulates credible treatment-time movements for the patient, and by regression it formulates a multiscale model that expresses the relationship giving the patient’s movements as a function of the corresponding changes in the tomographic projections. When the patient’s real-time projection images are acquired at treatment time, their differences from corresponding projections of the planning-time CT followed by applications of the calculated model allows the patient’s movements to be estimated. Using that estimation, the treatment-time 3D image can be estimated by transforming the planning CT image with the estimated movements,and from this, changes in the tomographic projections between those computed from the transformed CT and the real-time projection images can be calculated. The iterative, multiscale application of these steps converges to the repositioning movements. By this means, this method can overcome the deficiencies in limited-angle tomosynthesis and thus assist the clinician performing an image-guided treatment. We demonstrate the method’s success in capturing patients’ rigid motions with subvoxel accuracy with noise-added projection images of head and neck CTs.

This paper proposes a new bottom-up paradigm for detecting visual saliency in images and videos, which is based on scale space analysis of the log amplitude spectrum of natural images and videos. A salient region is assumed to be any region exhibiting a distinct pattern whose intensity, color, texture and motion is different from the rest of the image. Thus patterns which appear frequently as well as uniform regions are suppressed to produce salient region pop-out. We show that the convolution of the image log amplitude spectrum with a low-pass Gaussian kernel (at the proper scale) is equivalent to such an image saliency detector. A saliency map can then be obtained by reconstructing the 2-D signal using the original phase spectrum and an appropriately filtered log amplitude spectrum to produce pop-out. The optimal scale for each image feature channel (intensity, color, motion) is determined by minimizing the entropy of its saliency map. This produces four maps which are then fused by a weighted linear combination. Significantly, the approach does not require the setting of any parameters. We demonstrate experimentally that the proposed model has the ability to highlight small and large salient regions and to inhibit repeating patterns in both images and videos.

Principal component analysis (PCA) for various types of image data is analyzed in terms of the forward and backward stepwise viewpoints. In the traditional forward view, PCA and approximating subspaces are constructed from lower dimension to higher dimension. The backward approach builds PCA in the reverse order from higher dimension to lower dimension.We see that for manifold data the backward view gives much more natural and accessible generalizations of PCA. As a backward stepwise approach, composite Principal Nested Spheres, which generalizes PCA, is proposed. In an example describing the motion of the lung based on CT images, we show that composite Principal Nested Spheres captures landmark data more succinctly than forward PCA methods.

This paper evaluates the performance of different Markov Random Field (MRF) based stereo algorithms for cluttered scenes. These scenes are generated by randomly placing objects within a 3D volume. The scenes, which model natural cluttered scenes such as forests or bushes, contain many depth discontinuities and monocularly visible pixels. Widely used benchmark datasets do not contain stereo pairs with dense clutter, so we address how well existing stereo algorithms perform for such scenes. We use Expansion, Swap, Max Product Belief Propagation (BPM), Sequential Tree Reweighted Message Passing (TRW-S) and Sequential Belief Propagation (BP-S), all with different forms of data and smoothness terms. The results are compared with the ground truth disparity and energy.We found Expansion, TRW-S, and BP-M with the Potts model to work well for most scenes, in that the correct binocular correspondence is found for most points that are binocularly visible. We also found that the energy for the ground truth is much larger than what is found by the optimizers. This shows that there is room for improving the model for cluttered scenes.

We study the problem of approximating a solid with a union of overlapping spheres. We introduce a method based on medial spheres which, when compared to a state-of-the-art approach, offers more than an order of magnitude speed-up and achieves a tighter volumetric approximation of the original mesh, while using fewer spheres. The spheres generated by our method are internal to the object, which permits an exact error analysis and comparison with other sphere approximations. We demonstrate that a tight bounding volume hierarchy of our set of spheres may be constructed using rectangle-swept spheres as bounding volumes. Further, once our spheres are dilated, we show that this hierarchy generally offers superior performance in approximate separation distance tests.

The subject of this paper is a heuristic slicing algorithm for converting STL or PLY CAD data into boundary and fill paths for rapid freeze prototyping (RFP). The algorithm, developed for one commercial robotic system, can also be used to produce toolpaths for other rapid prototyping systems. The algorithm entails five steps: (a) geometry data and other control parameters are imported; (b) the geometry is sliced at several equidistant heights to form bounding paths; (c) contours for the scaffolding material are computed; (d) part and scaffolding paths are buffered in or out to account for deposition path width; and (e) fill paths are computed. A STL file of a 300 mm-high statue of James McGill is used as an example part for demonstrating the capabilities of the algorithm.

In the thermoforming industry, the heater temperature setpoints can be automatically tuned with Terminal Iterative Learning Control (TILC). This cycle-to-cycle control is used to adjust the heater temperature setpoints so that the temperature profile at the surface of the plastic sheet converges to the desired temperature. The robustness of a closed-loop system with this TILC algorithm is measured using the μ-analysis approach. A Genetic Algorithm (GA) is used to find the 2^nd order TILC controller parameters giving the most robust closed-loop system.

Our research aims to develop an anthropomorphic saxophone-playing robot; as an approach to understand the human motor control from an engineering point of view. In this paper, we present theWaseda Saxophonist Robot No. 2 (WAS-2) which is composed of 22 degrees of freedom (DOF). In particular, he functioning of the lips, fingers, tongue, oral cavity and lungs have been mechanically simulated to enable WAS-2 to play an alto saxophone. Furthermore, in order to ensure the accuracy of the air pressure control, a feed-forward control system with dead time compensation has been implemented.A set of experimentswere carried out to verify the effectiveness of the proposed system. From the experimental results, the range of sound pressure was increased and the air pressure control was improved.

A parallel cable-driven haptic interface is designed to allow interaction with any type of virtual object. This paper presents and analyzes computational methods for addressing the issues regarding human safety and control reliability using such an interface, thereby ensuring safe operations inside the virtual world. Four strategies are explored: sensor reliability, mechanical interference management, workspace management and human-robot interaction. This paper focuses mainly on the sensors’ reliability and workspace management algorithms for a parallel cable-driven haptic interface that imposes special requirements on the control architecture design. One challenging task is to develop efficient computational algorithms for hard real-time processes included in haptic display applications which improve safety without compromising performance.

The spatial double inverted pendulum actuated at the hip, but not at the foot, may be considered to be a model of standing creatures and robots. Moving inspace, as opposed to in-plane, poses new control problems which, for the most part, are still open. In this paper, a hybrid approach where an energy-shaping, passivitybased swing-up controller hands off the control to a linear-quadratic-regulator in the vicinity of the unstable upright equilibrium is proposed. A direct approach and a pre-compensated approach are described, discussed, and illustrated by means of examples in simulation.

In this paper, a closed-loop system to regulate plasma osmolality in patients with central diabetes insipidus is investigated. Using model identification techniques, we obtained a 3 ^rd -order LTI model of the renal/body fluid system from an 11th-order nonlinear system. A Smith Predictor and an \({\mathcal H}_\infty\) controller were developed. The effectiveness of the controller to reject a step disturbance in the plasma osmolality is examined. This paper shows the potential use of control theory in the context of central diabetes insipidus.

We examine the problem of learning a visual map of the environment based on discrete landmarks. While making this map we seek to maintain an accurate pose estimate for the mapping robots. Our approach is based on using a team of at least two (heterogeneous) mobile robots in a simple collaborative scheme. In many mapping contexts, a robot moves about the environment collecting data (images, in particular) which are later used to assemble a map; we view the map construction as both a knowledge acquisition and a training process. Without reference to the environment, as a robot collects training images, its position estimate accumulates errors, thus corrupting its estimate of the positions from which observations are taken. We address this problem by deploying a second robot to observe the first one as it explores, thereby establishing a virtual tether, and enabling an accurate estimate of the robot’s position while it constructs the map. We refer to this process as cooperative localization. The images collected during this process are assembled into a representation that allows vision-based position estimation from a single image at a later time. In addition to developing a formalism and concept, we validate our approach experimentally and present quantitative results demonstrating the performance of the method in over 90 trials.

The subject of this paper is the design and implementation of a robust dynamic feedback controller, based on the dynamic model of the four-wheel skidsteering RobuFAST A robot, undergoing high-speed turns. The control inputs are respectively the linear velocity and the yaw angle. The main objective of this paper is to formulate a sliding mode controller, robust enough to obviate the knowledge of the forces within the wheel-soil interaction, in the presence of sliding phenomena and ground-level fluctuations. Finally, experiments are conduced on a slippery ground to ascertain the efficiency of the control law.

New-generation, intelligent, powered wheelchairs promise to increase the mobility and freedom of individuals with serious chronic mobility impairments. And while rapid progress continues to be made in terms of the engineering capabilities of robotic wheelchairs, many projects fall short of the target in terms of ease of use, conviviality, and robustness. This paper describes the SmartWheeler, a multifunctional intelligent wheelchair, which leverages state-of-the-art probabilistic techniques for both autonomous navigation and user interaction modeling, to provide a novel robust solution to the problem of assistive mobility. We also discuss the use of standardized evaluation in the development and testing of such technology.

The future of space exploration will be increasingly surface-based and extended-duration. Planetary rovers, both manned and autonomous, will play vital roles in transporting instruments, astronauts, and equipment across rugged and unfamiliar surfaces. To enable this vision, it is advisable to deploy prototype rover vehicles in analog environments on Earth, in order to learn how best to use these tools. Devon Island, in the Canadian High Arctic, has been used as a proving ground for planetary rovers, due to its vast scale, variety of topography/geology, challenging lighting, lack of vegetation, existing infrastructure at the well-established Haughton- Mars Project Research Station, and wealth of interesting scientific mission objectives. In this paper we review the suitability of using Devon Island for the continued testing of planetary rovers; several examples of previously conducted tests are provided. We conclude that despite the typical logistical challenges associated with remote field work, Devon Island should be considered a strong candidate for ongoing rover field deployments.

In this paper we study a controlled Leader-Follower (L-F) flocking model (where the state of each agent consists of both its position and its controlled velocity) by use of the Mean Field (MF) Stochastic Control framework. We formulate the large population stochastic L-F flocking problem as a dynamic game problem. In this model, the agents have similar dynamics and are coupled via their nonlinear individual cost functions which are based on the uncontrolled Cucker and Smale (C-S) flocking algorithm. The cost of each leader is based on a trade-off between moving its velocity toward a certain reference velocity and a C-S type weighted average of all the leaders’ velocities. Followers react by tracking the C-S type weighted average of the velocities of all the leaders and followers. For this nonlinear dynamic game problem we derive two sets of coupled deterministic equations for both leaders and followers approximating the stochastic model in large population. Subject to the existence of unique solutions to these systems of equations we show that: (i) the set of MF control laws for the leaders possesses an \(\displaystyle\epsilon_N\)-Nash equilibrium property with respect to all other leaders, (ii) the set of MF control laws for the followers is almost surely \(\displaystyle \epsilon_N\)-Nash equilibrium with respect to all the other agents (leaders and followers), and (iii) \(\displaystyle \epsilon_N \rightarrow 0\) as the system’s population size, \(\displaystyle N\), goes to infinity. Furthermore, we analyze the MF system of equations for the leaders and followers with the linear coupling cost functions to retrieve similar MF equation systems in Linear-Quadratic-Gaussian (LQG) dynamic game problems.

In this paper, we present an overview of the work carried out in the Mechatronic Locomotion Laboratory at McGill University on a quadruped robotic platform, PAW. This robot features four springy legs with rotary actuation at the hips and driven wheels mounted at the distal ends of the legs. The robot was designed to explore hybrid modes of locomotion, where it makes use of both wheels and legs to achieve novel behaviors. As well, the robot’s simple construction allows PAW to exploit the dynamics of a mass-spring system to achieve gaits such as bounding, galloping and jumping. We begin by describing the basic design of the robot and its sensing capabilities. We then discuss several modes of locomotion that have been developed on the robot over the past five years. Specifically, results are presented for inclined turning and sprawled breaking achieved with the robot, as part of exploiting the leg capability in the rolling behaviors. This is followed by the presentation of the bounding gait implemented on the robot: the basic version and the intelligent version. The most recent addition to the robot’s repertoire of behaviors is a dynamic jump. We will discuss the main stages of the jumping maneuver and present the results of the jump. The paper concludes with a summary and discussion of our future goals for PAW.

This paper introduces a novel family of robots that consist of cablesuspended bodies whose motion is not fully constrained. The robots are underactuated and exhibit a pendulum-like behavior. Based on the dynamicmodel, a technique is proposed to allow the planning of point-to-point trajectories with zero-velocity landing by making the robot swing itself similarly to children on playground swings. A three-degree-of-freedom planar robot is studied as an example and a prototype of the robot and its controller are presented. Experimental results clearly demonstrate the effectiveness of the proposed control technique. Underactuated cable-suspended robots have the potential to lead to low-cost solutions in applications that require the performance of point-to-point trajectories in a large workspace.

Among other applications, accelerometer arrays have been used in crashworthiness studies to measure the acceleration field of the head of a dummy subjected to an impact. In previous analyzes, the centripetal component of the rigid-body acceleration was estimated linearly from point-acceleration measurements, that is, by considering the quadratic products of the angular-velocity components as independent variables. Although this assumption greatly simplifies the estimation process, it has two drawbacks: (i) it raises the minimum number of accelerometers from nine to 12, and, when more than the minimum number of accelerometers are available, (ii) it ignores some of the constraints between the kinematic parameters, which would otherwise help in filtering the data. In this paper, we solve the nonlinear problem of estimating the rigid-body acceleration field from point-acceleration measurements. To this end, we partition the associated system of equations into two subsystems, one linear, the other nonlinear. The nonlinear subsystem of three equations in three unknowns represents three quadrics in 3D space, whose intersection contains the rigid-body angular velocity. This intersection is computed using a readily-available technique, which yields eight closed-form solutions to the problem. A criterion for the selection of the proper solution is given. The proposed nonlinear method should prove useful when the number of accelerometers is limited, or to improve the robustness of an array of 12 or more accelerometers by taking into account the constraints between the quadratic terms of the angular velocity.

The subject of this paper deals with the singularity analysis of a sixdof three-legged parallel manipulator for force-feedback interface. To this end, a geometric condition for the manipulator singularities is obtained by means of Grassmann-Cayley algebra; the parallel singularities of the manipulator are computed using Jacobian and Gröbner basis. As a result, the algebraic relations of the singularities satisfied by the orientation variables are reported. Finally, the parallel singularities of the manipulator are plotted in its orientation workspace.