Machine Vision and Mechatronics in Practice

This paper describes new research and development currently being undertaken to advance the automation capability of mining equipment in underground coal mining operations. The research effort targets the development of new inertial navigation solutions that aim to increase roadway development rates and provide a safer working environment for underground mine personnel. Specific developments have been undertaken to develop and integrate navigation, remote control, and visualisation technologies. Results obtained so far show that the navigation solution developed can provide the accuracy necessary for production-level underground mining.

This paper documents an application of a scanning microwave radar system that has been developed by the CSIRO Mining Technology group as an aiding sensor for machine guidance in a mining environment. In this application, radar imaging is considered as a vital alternative to other technologies such as optical imaging based on laser technology, because the imaged environment is characterized by high levels of airborne and surface dust. In this case, the microwave radar offers a higher level of immunity against fine particle interference than an optical system. This is an attractive solution provided that a lower resolution image is acceptable. The developed scanning radar system combines a mechanical pan-tilt scanning platform and a 26 GHz monostatic radar unit. The prototype system is employed in the field for the purpose of collision avoidance on a shiploader boom. Raw data from this unit is processed to allow comparison with corresponding laser scanner point data, initially for the 2D case of measuring the distance-to-deck from the shiploader boom. The 2D results are encouraging. Based on these positive results, a software interface featuring a 3D display of detected targets is developed.

This paper presents results obtained through the development of a terrain mapping system, which combines millimeter wave and laser scanners to provide enhanced mining situational awareness capability. A comparison of the use of laser and radar data for the purpose of creating Digital Terrain Maps (DTMs) and object pose estimation is given which includes a comparative evaluation of terrain mapping data obtained from prototype millimeter-wave radar and several commercially available 2D scanning lasers mounted on large rotating excavation machinery. The registered laser data is compared with the results of a terrain model obtained through stereo vision, and, subsequently, with the radar data.

This paper describes a framework for experiments in Human-Computer Interaction, using immersive virtual reality, computer vision and other sensors, and remote collaboration. The proposed framework is demonstrated in a number of applications.

This paper describes the development of an automation system for longwall mining machinery which delivers centimeter-order position accuracy with no drift. This system has been developed using high performance inertial navigation technologies coupled with novel aiding strategies to eliminate integration drift.

This paper describes the challenges and opportunities in taking a special purpose robotic orthopaedic system from research through to a spin-off company with clinical application and then on to acquisition.

A mechatronic surgical micro-drill applied in practice has enabled new techniques for measurement in otology to understand the dynamic response of the cochlea hearing organ. In the interest of reducing trauma and to reinforce preservation of residual hearing during surgery, results point to choices in technique as mechatronic devices offer significantly less disturbance than conventional surgical tools used in cochlear electrode implantation. The cochlea transforms mechanical disturbances induced from incident ambient sound to electrical neuro-signals interpreted in the brain as sound, and is a closed volume of fluids and delicate membranes. The mechatronic drilling technique can expose consistently sized membranes at the base of a hole drilled through the bone tissue shell of the cochlea. The dynamic deflection of membrane windows, in response to fluid pressure transients, enables contrast between intra-cochlear disturbances at different locations within the cochlea, and between normal hearing disturbance amplitude and disturbances induced by surgical processes.

urveillance with the help of an aerial robot is an interesting research topic, studied also in this work. With the help of a lightweight camera and a versatile processing platform, target recognition can be achieved with very low cost hardware. This paper considers Quanser Qball x4 system with Open TLD toolbox operating in the ground station. The Qball x4 system can provide flight without human intervention and during the flight, obtained video sequence is transmitted to the ground processing host computer, which runs open TLD module. The operator marks the object to be tracked, and then the software marks the video image whenever it finds the predetermined patterns in the video stream. The real time applications show that the integration of such modules provides very good results in recognizing stationary and moving targets.

In this study Active Disturbance Rejection Control (ADRC) is implemented for the attitude dynamics of the quadrotor flying platform. Quadrotor model with a 2 DOF manipulator is constructed. External disturbances, manipulator dynamics and internal plant dynamics are treated as total disturbance which are estimated and rejected in simulation environment.

Haptic devices mimic force or torque responses of physical systems. In active devices, the synthetic responses are usually produced by DC motors, due to their wide range availability and ease of control. On the other hand, simulating rigid collisions require large motors, at the expense of size and weight, or gearboxes at the expense of transparency. Viscosity of magneto-rheological fluids (MRF) increases, up to the point of becoming a viscoelastic solid, when they are subjected to a magnetic field. Due to this property, they are used in adjustable dampers, where the damping force is controlled by varying the intensity of the magnetic field via an electromagnet. The use of MRF technology in passive or semi-active haptic devices is an attractive research area, in which, a number of damper designs have been proposed over the last decade. This paper investigates a linear MRF damper for haptic finger grasping applications. The damper incorporates an orifice area which is affected by a position controlled permanent magnet. Proximity of the permanent magnet to the orifice area manipulates the viscosity of the MRF; hence changes the reaction force of the damper.

There are several different ways to create and control virtual 3D (three-dimensional) models of robots, however, most of these methods can only be implemented or understood by experts with years of extensive experience in 3D graphics programming, 3D mathematics and a plethora of advanced skills in the areas of solid modelling and 3D animation. This paper presents a brief history and summary of the state-of-the-art in 3D game development tools and technologies which can be used to develop realistic looking graphics for developing user interfaces and robot control programming tools. It also presents a simple and easy-to-learn kinematic modelling and 3D simulation process using a 4 degree of freedom (4-dof) articulated robot leg for an amphibious walking and swimming robot that is currently being designed by the authors. This 4-dof robot leg will be used as an example or case study to demonstrate an effective method for motion control, animation and simulation. Also described are popular software tools and essential skills needed to create a simple 3D simulation program. The source code of the 3D simulation software for the 4-dof robot leg is listed and described to help readers apply such methods to other robot designs, devices and complex machinery.

This paper justifies the need for more research in the area of ‘assistive robotics’ and it describes the design and performance of a manipulator arm and gripper, mounted on an electric scooter, that can enable frail elderly and people with disabilities to collect and retrieve heavy objects, such as shopping items, located on high shelves. Known by the acronym ‘ESRA’, short for ‘Electric Scooter Robot Arm’, this device can be attached to a mobile vehicle like an electric scooter or an electric-powered wheelchair. This paper discusses the basic mechanical design of the ESRA manipulator and gripper, joint control, important design tools and failure analysis that were used for its development, and overall performance of the prototype arm. The object retrieval performance of this manipulator can also be enhanced using machine vision techniques for automatic object detection and gripper guidance. Different end-effector tools can be mounted on the end of the ESRA to enable the operator to perform a wide variety of common household chores and manipulation tasks.

The new structure of spatial parallel robots from Delta robots family is elaborated in this research. The proposed novel mechanism like ordinary Delta parallel mechanism has three degrees of pure translational freedom, but the position of robot’s three active joints relative to each other is one difference between this mechanism and Delta parallel mechanism, which has caused the change in geometry of platforms, and it shapes the asymmetrical structure in the robot mechanism and its workspace. Another difference arises from an architectural optimization methodology by consideration of mixed performance index which has utilized in this mechanism for reaching a better compromise between the dexterity of manipulator and its workspace volume. Inverse dynamic modeling is performed based on Lagrange formulation. The PD and PID controllers of Computed Torque method (C-T) usually need manual retuning to make a successful industrial application, particularly in the presence of disturbance. In the present paper, we study feasibility of applying fuzzy supervisory control for PD and PID used in C-T method. Numerous computer simulations demonstrate the effectiveness of proposed control method in comparison with ordinary C-T method.

This paper presents investigations into the development of hybrid control schemes for input tracking and anti-swaying control of a rotary crane system. A lab-scaled rotary crane is considered and the dynamic model of the system is derived using Euler-Lagrange formulation. To study the effectiveness of the controllers, initially a collocated proportional-derivative (PD) control is developed for horizontal angle position control of rotary crane. This is then extended to incorporate input shaper control schemes for anti-swaying control of the system. The positive input shapers with the derivative effects are designed based on the properties of the system. Implementation results of the response of the rotary crane with the controllers are presented in time and frequency domains. The performances of hybrid control schemes are examined in terms of the level of input tracking capability, swing angle reduction, and time response specifications in comparison to PD control. Finally a comparative assessment of the control techniques is discussed and presented.

Large mobile irrigation machines are self-propelled sprinkler irrigation systems which farmers are rapidly adopting due to the high precision of the irrigation application. Although it is highly desirable that control systems be used with such machines to both optimise the irrigation water volume applied to field crops and optimise water use efficiency, there are difficulties in applying classical control techniques. These are caused principally by the very slow speed of crop growth-response and stress-response dynamics; but in addition characteristics of the plant which are poorly known and in-field sensors which provide only sparse, low-quality data for feedback control.

This paper outlines the operation of large mobile irrigation machines, analyses the limitations in the application of classical control approaches for their optimal use, and describes the methods that have been used to implement whole-system control via alternative (adaptive) approaches. These involve accommodation of sparse and unreliable input data and the application and evaluation of a range of irrigation volumes on different sub-areas of the field as on-the-go local system identification.

A method is investigated for damping the oscillation of a very wide agricultural spray rig. When mechanical dampers are used, they couple the rolling disturbance of the vehicle into the tilt of the spray booms, so an alternative technique is desirable. A method whereby the booms are driven in a manner analogous to a tightrope walker’s balancing pole is shown to be effective.

At present asphalt pavement defects are distinguished subjectively. The recognition depends greatly on human vision, which can bring large variations for micro-wave heating control. The paper adopts machine vision aided locating to create accurate heating control data that is objective and independent of human to overcome the deficiency. Firstly, the micro-wave heating control system (MWHCS) based on machine vision is presented. Then, the algorithms of cubic polynomial warping and reprojection transformation are applied to rectify the image distortion. Thirdly, the automatic and manual methods to create heating control data are discussed. At last, an ecnomical driving circuit design matching with the heating data is demonstrated. Product testing proves that the proposed heating control method with machine vision is prompt and accurate in locating defects, also quite effective in energy saving.

This paper describes the conceptual design and operating principles of an oscillating-foil propulsion system for an unmanned underwater vehicle called TURTLE (‘Tele-operated Unmanned Robot for Telemetry and Legged Exploration’), currently under development. This UUV (‘Unmanned Underwater Vehicle) will be designed to be a 6-legged swimming and walking amphibious robot, fitted with foils (or flat fins) which can be manipulated with several degrees of freedom to produce highly efficient underwater propulsion forces. The legs will each have four degrees of freedom, of which the fourth is rotation of a foil that is fitted to the ’shin’ to provide propulsion for swimming. By manipulating the movements and rotations of this foil, propulsion forces can be generated to implement a variety of swimming modes, each with its own advantages and disadvantages. The foils attached to the fins allow the main body to be controlled in all six degrees of freedom. It will also be an amphibious robot that will be able to transition between swimming mode and walking mode, for walking on an underwater surface or over dry land if power considerations permit. It must be powerful and strong enough to support itself and light payloads while walking over rough or undulating surfaces commonly found on a beach. The mechanical design will allow the absolute position and orientation of the body to be accurately controlled relative to the ground surface, whether above or below water, for the purpose of precision control of onboard tools and sensors. The space frame construction method keeps water drag low and allows large scale, strong, rigid structures and manipulator limbs (or links) to be built. Space frames also keep material cost, weight and actuator energy usage to very low levels. Such lightweight and energy efficient robots will be useful in many practical applications, such as oil and gas exploration, drilling, mining, construction, automated agriculture, military transport and space exploration.

In engineering applications, the very precise control of parallel manipulators to track the desired trajectory has received a great deal of attention. This paper presents a new methodology to investigate this problem based on the Adaptive Neuro-Fuzzy Inference System (ANFIS) controller for three degrees of pure translational freedom of the spatial parallel manipulator by employing the architecture of a modified DELTA robot. At first, the mobility and velocity analysis of the manipulator is carried out in details. The inverse dynamic modeling is performed based on the principle of virtual work. Then, the control algorithm using ANFIS method is implemented and simulation results show the better performance in comparison with computed torque method.

In this study, a low-cost navigation system is proposed for enhancing highway traffic safety in adverse environments such as fog or rain. An instrumentation system is proposed to be installed onboard each car. The instrumentation system consists of a microcontroller, a GPS receiver, an IMU, a digital compass, and a Zigbee transceiver. The microcontroller system samples the measurements of the GPS receiver, the IMU, and the digital compass to provide a vehicle navigation solution at a 50Hz frequency. This navigation solution would provide the position, velocity, and attitude of the vehicle which would then be passed on to all other vehicle within a close range using the radio communication system. When each vehicle has position and velocity data for its surrounding vehicles it can warn the driver if another vehicle is too close or approaching too fast. Experimental results of the proposed system are shown. First, single vehicle tests, both static and dynamic, are used to assess the accuracy of the low-cost navigation systems against a commercial solution. Finally, experimental results of two-vehicle tests are shown.

EEG-controlled mechatronic and robotic systems provides additional flexibility and control option for both the disabled and able body persons. Providing adaptive control option through EEG as the source control signal requires efficient embedded technology system for EEG feature and artefact extraction toward robot motion control. The encoding and decoding of EEG signal allows for efficient EEG artefact extraction and selection in embedded systems.

This paper presents the decoding and encoding of expressive EEG signal recorded from physiological expressions. The EEG signals are generated from physiological expressions and provide the base signal in the analysis of EEG signal. Encoding and decoding of the Expressive EEG signal allowed for further development of robotic system motion control commands.

Illumination variation is a challenge problem at face recognition since a face image varies as illumination changes. In this paper, it is reviewed the illumination variation methods in the state-of-the-art such as the single scale retinex algorithm, the multi scale retinex algorithm, the gradientfaces based normalization method, the Tan and Triggs normalization method and the single scale weberfaces normalization method. The face recognition is performed by using Principal Component Analysis (PCA) in MATLAB environment. AR face database is used for evaluating the face recognition algorithm using PCA. The distance classifier called as Squared Euclidean is used. Experimental results are comparatively demonstrated.

This paper discusses the benefits of using competitions in the teaching activities for a 1^st or 2^nd year mechatronic engineering subject that teaches microcontrollers and/or mobile robotics using a ‘hands on’ PBL (Problem Based Learning) approach to learning and assessment. It describes, in detail, effective competition design philosophies and guidelines for creating different types of robotics competitions (and their rules) which help to keep students highly motivated and engaged in their learning and self-education. A ‘hands on’ one-semester microcontroller programming subject is used to illustrate three different case studies (competitions) that were implemented. This particular subject achieved very high satisfaction marks in student surveys and attracted a great deal of praise from students. The same teaching principles and similar kinds of technical competitions can also be used to achieve successful learning outcomes in other kinds of technical or engineering related subjects. This paper also describes general principles for designing an effective lab based course structure and how technical concepts and materials can be presented, learned quickly, and assessed in order to produce very interested and highly motivated students who are keen to teach themselves practical real world skills with minimum stress and maximum enjoyment. The case studies include a robotic racing car competition, a multi-player box grabbing contest over a rocky obstacle course and a ‘robot wars / robot sumo’ style competition that was televised nationwide in Australia.

The demands of modern manufacturing require production systems to be flexible and adaptable. These systems must be able to respond to changes in product demand and the frequent introduction of new products and technology. This paper presents the development of an Open Architecture Controller (OAC) for reconfigurable Numerically Controlled (NC) machine tools. The research involved the development of a novel scalable solution that parallels the modular mechanical nature of reconfigurable machinery. A one to one mapping between mechanical and electronic systems was achieved by a modular distributed axis drive control system and a PC based OAC that had the ability to manage drive controllers from multiple vendors. A class based C++ implemented Graphical User Interface (GUI) allowed the user to configure the functions of the OAC to match a reconfigurable machine structure. The OAC was tested on a reconfigurable machine and the results are presented.

An investigation is presented of a low cost approach to the measurement of two and three dimensional objects using a flatbed scanner and image analysis software. Conventional measurement using relatively low cost instruments such as micrometers and vernier callipers can be time consuming and requires operator skills which result in higher overall costs. The increasing resolution and decreasing prices of flatbed scanners introduces the possibility of their use as a low cost alternative to traditional manual measuring. To investigate this, a simple dimensional measurement technique was developed using an unmodified, then a modified, flatbed scanner, a standard PC, and software. A dimensional accuracy of ±0.05 mm was achieved with a modified flatbed scanner system for slip gauge samples of nominal thickness 10mm and 5mm.

This paper proposes an intelligent soft starting method for hub motors in applications to electric vehicles. This method proves a good linearity to rising current of hub motor at the starting point. So, the negative effects of the current peaks could be prevented. At the same time, a smooth rise has been ensured in the motor speed. Current of hub motor and motor speed have been used as feedback data to the microcontroller. The used microcontroller is an ARM based 32 bit microcontroller that is necessary to handle complex mathematical processes of fuzzy type II method. The proposed system has been tested in various load and speed conditions. Also, PID, fuzzy logic and fuzzy type II methods tested in the same conditions to show which one of them gives the best result. The results of the experiments have been taken to the Matlab environment to visualize and analyze them.