Realization of heavy object transportation by mobile robots using handcarts and outrigger

Mobile robot-based object transportation method is important and need to be developed for use in cargo transport, home moving, and other similar applications. The robots have to be compact in home and office environments, because these places are generally narrow. The objects to be transported can range broadly from small to large ones. If the robots can transport heavy objects that are particularly difficult for one person to handle, they will greatly help in reducing the physical burden on people. In other words, transport in the hypothesized robot environments demands smaller mobile robots and handling of heavy objects.

Presently, the weight of objects that a mobile robot can hold (called “payload”) is not large. For instance, the payload of the mobile manipulator KUKA youbot [1] is 0.5 kg while the machine itself weighs 26.3 kg; hence, the payload/self-weight ratio of approximately 0.02. The WAM Arm from Barrett Technology [2] has a payload of 4 kg against a self-weight of 25 kg, with a payload/self-weight ratio of approximately 0.16. Thus, increasing the weight of the transported object increases the sizes of the robots that can handle it. As already stated, the hypothesized environments are narrow, and they are unsuited for deploying a large robot. One way of effectively overcoming this problem is coordinating multiple small mobile robots. In other words, it is important to develop a method for transporting large objects by using multiple small robots.

Many studies have been carried out on transporting objects using multiple mobile robots.

Khatib discussed mobile manipulation with the concept of the virtual linkage [3]. Yang et al. proposed a leader–follower type control methodology of several robots for transporting a single object [4]. Hirata et al. also solved the similar problem with Ref. [4] but they adopted the concept of the virtual 3-D caster [5]. Liu et al. presented the transportation system with a gripper robot and a lifter robot [6]. Doger et al. realized assembling of large box-like objects by using a kind of hierarchical planning [7]. These studies deal with transport by coordinated grasping. Coordinated grasping means multiple robots lifting and supporting a single object (Fig. 1a). The type of robot widely used is a mobile manipulator, which is a mobile robot equipped with a multi-degrees-of-freedom manipulator. However, as stated above, the payload of a single mobile manipulator is low. With coordinated grasping, assuming n robots are used, the payload is at the most only n times the payload of one robot. Yonezawa et al. [8] described the development of a system consisting of multiple specialized robots that lift automobiles using a lifter. Wurman et al. [9] proposed a warehouse management system called KIVA system, where many small mobile robots lift up and transport movable storage shelves to improve productivity in the warehouse. These studies limited the type of objects that can be transported to only automobiles or storage shelves.

Okawa et al. [10] proposed control strategy of a mobile robot that pushes a box from one location to another. Rus et al. [11] showed the control method of multiple robots that cooperatively push a large object such as furniture to its destination. Kube et al. [12] presented a method for moving a large object by multiple robots whose location is initially unknown to the robots, where the algorithm was proposed in reference to the behavior of ants. These studies deal with transport methods based on coordinated pushing. In such transport methods, mobile robots push an object while it remains in contact with the floor (Fig. 1b). Transport of objects based on pushing does not require mobile robots to support the full weight of the object transported; hence, the payload in this case is larger than that in the case of coordinated grasping transport. However, because of friction between the object and the floor surface during transport, continuing the work becomes difficult. In brief, the problem is the impact of the payload of mobile robots on the coefficient of friction between the floor of the work environment and the transported object. Several studies discuss other pushing transport methods wherein the robots use bars or ropes as tools to supplement the pushing work to transport the object. Yamashita et al. [13] proposed a planning strategy for manipulating multiple objects at a time. Donald solved a similar problem with Ref. [13], by using several manipulation skills [14]. Both reports mainly describe the planning and control of mobile robot operation, without referring to the weight of the object or the coefficient of friction with the floor.

The concept of graspless manipulation of objects were presented in several studies, as a method of manipulating them. Sawasaki et al. [15] proposed a control algorithm for tumbling operation by a robot. Aiyama et al. [16] presented methodology of pivoting manipulation by robot fingers. How to select appropriate operations with sensor feedback was proposed in Ref. [17]. Maeda et al. proposed a method of planning of general graspless manipulation by multifingered robot hands [18]. These papers discuss methods of varying the position and orientation of the objects as they remain in contact with aspects of the environment, by tumbling, pivoting, or tilting them instead of grasping them. This method of transporting objects was intended for use by fixed-type manipulators. Yoshida et al. [19] extended the concept of the graspless manipulation to the motion control of humanoid robots that transport a large object, and introduced obstacle avoidance behavior to the humanoid robot while transportation [20]. However, these operations are not realized by small mobile robots.

Sakuyama et al. [21] reported on the transport of a large object by combining the tilting operation with tools. Here, handcarts are used to carry the object (Fig. 1c). Figure 2 shows the specific transport method. Here, two robots are used: a robot used to tilt the object and a handcart transport robot. The object-tilting robot tilts the object, and the handcart transport robot inserts two handcarts into the space under the object. This operation is performed twice to load the object to be transported onto handcarts. This method reduces the load on the robots because the robots do not support the full weight of the object. However, there is a risk that during the tilting operation (Fig. 2a, c), the reaction force of the object on the robots and the impact of the friction of the robots and handcarts on the floor will tilt a robot or cause a handcart to slide, preventing successful transport of the object. Because of these risks, the weight of objects that this system can transport are still insufficient.

Past research of this kind has shown that heavy-object transport by using multiple mobile robots has still not been successfully achieved.

The problems of tilting of robots or sliding of handcarts during transport by a group of small mobile robots using handcarts have been described in the previous subsection. The objective of this research is to resolve these problems to realize the transport of heavy objects by using multiple small mobile robots. An outrigger device is employed to deal with the first problem, which is tilting, and a handcart lock device is used to deal with the latter problem, which is sliding. Both devices have to be used only when necessary; hence, we propose a mechanism and a control method capable of ensuring that a device is activated only when necessary. To prevent sliding, it is necessary that when the robot is pushing the object to be loaded on the handcarts, the handcarts can move smoothly. When the object is tilted, the handcarts must be locked. To prevent tilting, when the robot is pushing the transported object, the outrigger is operated, but if the robot is moving, the outrigger would be an obstruction.

The paper is composed as follows. In “Methods” section, the problems that this research will attempt to resolve are described and a brief explanation of the proposed system is presented. “Implementation” section describes the mounting of the transport system, and “Experiments” section describes the experiments. Finally, in “Conclusion” section, the conclusions and future prospects are presented.

This paper assumes the following work environment

The weight of the transported object is set according to Ref. [22]: “Efforts shall be made for it to be no more than approximately 40% of the weight of an object handled manually by male workers 18 years old or older.” According to Ref. [23], the average body weight of an adult Japanese male in 2013 was 65.8 kg. Assuming that if the robots can transport an object whose weight is 40% of the person’s weight (approximately 26 kg), they can contribute in reducing the physical burden on people handling heavy objects, the weight of the hypothetical transported object has been set at approximately 30 kg. The dimensions of the object were roughly calculated as 1.0 m height, 0.5 m width, and 0.5 m depth, which are assumed to be the same as that of a person. The working environment is assumed to consist of a flat floor, which is the normal home environment. Considering that the corridors in office buildings are at least 1.2 m width [24], the size of the mobile robots was set as a size that would fit in a cube with a side length of 0.6 m, which is approximately half the corridor width, considering that a robot would meet and pass people.

The transport procedure using handcarts was set as in Ref. [21]. The following is the procedure:

The problems during the loading process (1) are, as stated earlier, the tilting of the robot or the sliding of the handcarts. During transport (2), the friction with the floor has to be reduced while the transported object is on the handcarts so that the mobile robots can transport the object by pushing. Control of the robots when they are pushing has, as stated above, already been proposed, and can be adopted. The steps of the unloading process (3) are assumed to be the steps used to load the object performed in reverse.

As stated above, the scope of this study extends from loading the object on the handcarts (1) to pushing it (2).

This paper describes the successful development of a transport system consisting of an object-tilting robot and a handcart transport robot that transport objects using multiple handcarts. Each robot performs a specific task: tilting the transported object to create a space underneath it, and transporting handcarts and inserting them in the space under the transported object.

For transporting the object, first the object is tilted by the object-tilting robot. Tilting is done twice, and each time, two handcarts are inserted by the handcart transport robot. After all the four handcarts are inserted and the loading of the transported object has been completed, the tilting robot moves to its specified location, and pushes the object to transport it (Fig. 3). It should be noted here that the handcarts are inserted by the handcart transport robot.

As stated earlier, the transport method using handcarts and mobile robots faces the following two problems. One problem is the sliding of the already inserted handcarts during the second tilting work. This is a result of the low friction force between the handcarts and the floor surface. If the friction force is too high, it will be difficult to push the object. In brief, during the tilting operation and during the pushing operation, there is a big difference in the required coefficient of friction between the handcarts and the floor surface. The second problem is the danger of tilting or sliding caused by the reaction force generated while the robot is tilting the object. Thus, while operating the object, the robots are fixed in position, and when moving, they are released. To address the two problems, functions that fix and release the object according to the tasks that both robots are performing is required. Regarding the former, handcarts are equipped with a controllable lock mechanism that can lock or unlock the handcarts. The latter problem is resolved by using an outrigger.

For determining (a) the location and the direction in which the mobile robots push the object in the tilting operation, and (b) the location and the direction in which the handcarts are inserted in the space under the object, we adopted the method proposed in Ref. [21], and they will not be discussed in detail in this paper.

The process of transporting a cuboid object with four handcarts is explained here, but this proposed method is not necessarily limited to four handcarts. The proposed transport method can also resolve problems when using n handcarts to transport general shaped objects.

Before designing hardware configuration, mechanical analysis is conducted in object tilting process. Meaning of variables and constants is explained in Fig. 4 and Table 1. Here, we assume that the direction of the force from the end effector is the tangent direction of the arc centered on the axis of rotation of the object [21], and we assume that the end effector is position-controlled.

where

Inequation (1) shows the condition for not overturning of the robot. Inequation (2) shows the condition for not sliding of the robot. By using these inequations, we can analyze the relationship between the configuration of the robot (such as the length of the outrigger, weight and center of gravity of the robot) and the characteristics of the object (such as the size, weight, and the center of gravity). Here we do not consider the sliding of the handcart.

The hardware used by this transport system include the object-tilting robot, handcart transport robot, and handcarts. The robots use Pioneer 3-DX from Adept MobileRobots as their platforms.

1. Object-tilting robot: Fig. 5 shows the object-tilting robot developed for this study. The end effector that tilts the object has two degrees of freedom, one in the vertical direction and the other in the horizontal direction. The outrigger is installed on the back of the robot where it can be switched between housed and deployed states. A rubber sponge is attached to the tip of the outrigger, increasing friction with the floor to improve the device’s gripping capability. The overall dimensions of the robot are 1.02 m height, 0.38 m width, and 0.59 m depth when it houses the outrigger, to fit within the required 0.6 m. With solving minimal length of L that satisfies inequations (1) and (2), the outrigger was designed with L = 0.5 m, while assuming h = 1, W = 0.5, H_c = 0, W_a = 0.35, W_o = 0.45, m_obj = 105, m_rob = 29, H_r = 0.3, μ_w = 0.8, and μ_o = 0.8. These values were set for considering standard situation. Here m_obj = 105 was obtained by multiplying 30 kg in Subsection 2.1, by the safety factor of 3.5, which was set with reference to the existing hypothesis concerning the safety factor [25].

2. Handcart transport robots: Robots with parallel grippers installed on their front to hold the handcarts were deployed (Fig. 6) as handcart transport robots. The grippers opened horizontally to a maximum distance of 2.05 × 10⁻³ m, and the maximum transportable weight of each gripper is 2.5 kg. This means that normal coordinated holding by two mobile robots can transport an object with a maximum weight of 5 kg.

3. Handcarts: Two approaches can be considered in designing the handcarts: (a) to lock caster wheels mounted on the handcart when necessary, and (b) to push the floor with large frictional material when necessary. We select the method (b) for simplicity in design [we do not need to lock all wheels respectively in (b)]. As for selection of actuator to push the floor, an air cylinder and a plastic bottle are used because the friction of the handcarts can be easily changed just with opening or closing an air valve. As for the material with large friction, nonskid board instead of spikes is selected because flat floor surface is assumed as the work environment. Eventually, as shown in Fig. 7, a nonskid board installed on the bottom surface is pressed against the floor surface by an air cylinder drive, fixing the handcart. This cylinder drive can be utilized with about 10 times from the viewpoint of capacity of the plastic bottle, and this handcart system can be used even with loading and unloading process of the object. Equipped with a microcomputer and a wireless communication device, the handcarts can be locked or released any time. The allowed load of one handcart is 57.6 kg, which can fully support the weight of the object assumed as the object transported in this research. Figure 8 shows a handcart equipped with a controlled locking mechanism.

When the effectiveness of the proposed outrigger and the handcart transport system were theoretically calculated using inequations (1) and (2), the result indicated that the maximum weight of the object that can be tilted when this system is not adopted is about 37 kg. Comparing this with the 105 kg that can be tilted when the system is adopted shows that adopting the system as proposed increases the object weight that can be tilted by about 2.8 times. If the outrigger is longer than 0.5 m, the maximum weight that can be tilted increases, but at the same time, the area occupied by the robot increases. Hence, it is necessary to consider this trade-off while designing the outrigger.

To realize the proposed transport system, it is essential for the robot to recognize the object and to estimate its position and orientation. In this research, the robots need to measure the transported object and the handcarts. It is necessary for the object-tilting robot to tilt and push the object, and is necessary for the handcart transport robot to operate the handcarts.

In this paper, by using a camera to take a picture of and recognize a marker attached to the object, estimation of the object’s position and orientation is realized. For marker recognition, ARToolKit [26] is used. ARToolKit is an augmented reality building tool based on image processing that is released as an open source library. Recognition is done using black square markers (Fig. 9) with recognition use patterns marked inside them to calculate in real time the position and orientation in the camera’s coordinate system. The letters inside are used only for monitoring and not for robot control.

As stated earlier, the methods of deciding the position and direction to push the object, its tilt angle, and the location of installation of the handcarts are determined by using the method shown in Ref. [21]. Summary of the method is as follows: in calculating the push position and the direction upon the object, several constraints are considered: for the object, pushing position and direction need to be determined for the object not to slide on the floor. They also need to be within the movable range of the end effector. By setting the performance index as to minimize the risk of slip between the object and floor and between the object and the end effector during tilting of the object, the push position and direction on the object are determined. The location of installation of the handcarts under the object is determined to minimize the risk of overturning of the object.

In this paper, the trajectories of the robots to go around the objects and to go between the object and handcarts are calculated in advance.

To verify the effectiveness of the proposed system, a transport experiment using an actual object, robots, and handcarts was performed. In the real world, it will be clarified whether or not the object can be operated when there are sensing errors, control errors, looseness and softness of robot mechanism, or various other errors.

1. Transported object: The rostrum shown in Fig. 10a is the transported object. This object weighs 35 kg, and its dimensions are 9.7 × 10⁻¹ m height, 6.0 × 10⁻¹ m width, and 4.5 × 10⁻¹ m depth. The front and sides of the object are covered with panels, while there is no panel on its back, and its inside is empty. Therefore, its center of gravity is off the centroid and slightly towards the front. Therefore, when the object is tilted, the risk of overturning of the object varies according to the surface pushed. Considering this fact, in this experiment, the first tilting operation by the object-tilting robot was done from the front of the transported object, and the second tilting operation was done from behind (Fig. 10b).

2. Experimental instruments: The sensors used by the object-tilting robot and the handcart transport robot were 2 million pixel image web cameras, UCAM-C0220FB made by Elecom. Each sensor was installed on the top of the robots. To monitor the experiment results, another camera (called the “side camera” in this paper) installed on one side for transport work.

3. Experiment conditions: As experiment conditions, the work environment was the same as the assumptions described in "Assumptions" subsection. Figure 11 shows the transported object, object-tilting robot, handcart transport robot used for the object transport experiment, and initial locations of the handcarts.

4. Experiment method: The object transport experiment was done assuming a case where during the object-tilting operation, the outrigger of the object-tilting robot is deployed.

The change in the position of the object-tilting robot during tilting was measured. ARToolKit described above were applied to the side surface of the robot, front and back of the object, and to the fronts of the handcarts and these were measured by the cameras installed on the robots and the camera installed to the side, to measure the relative positions and orientations of the cameras and markers. Because the locations on the robots and the transported object where the markers were placed were already known, it was possible to measure the positions and orientations of the robots and the object.

5. Analysis method: From the data obtained by the side camera, the measurement data of change in the positions of the object and the tilting robot were obtained and analyzed. The frame rate during measurement was set at 30 fps.

Figure 12 shows the object transport flow using the outrigger. (a) is the initial condition and (b) shows the object-tilting robot performing the first tilt with its outrigger deployed. At this time, the robot is pushing at a height of 9.1 × 10⁻¹ m. (c) and (d) show the handcart insertion robot inserting handcarts in the opened space. After two handcarts have been completely inserted, the handcarts are locked. Next, (e) shows the object-tilting robot with its outrigger housed after completing the first tilting work, (f) shows the robot moved to the opposite side of the object, and (g) shows the robot with its outrigger redeployed as it performs the second tilting operation. (h) and (i) show the handcart insertion robot again inserting two handcarts and completing the second tilting operation. After the handcart locks are released and the object-tilting robot has housed its outrigger, in (j), the robot moves in the direction it pushes the object in, and in (k) and (l), it transports the object. This shows that object transport can be realized in this way. The maximum handcart position error caused by sensing errors or control errors was 6.0 × 10⁻² m, but even in a case with this degree of error, transport is fully achieved. This shows that the performance levels of the measurement system and the control system proposed are sufficient.

Next, the change in the position of the robot during the first and second tilting operations are analyzed. These correspond to the period of the time between Fig. 12a, b, and the period of the time between Fig. 12f, g respectively.

The moving average of 11 points set by combining the measured value at the time when the object-tilting robot is at a certain position and the measured values at 5 times before and 5 times after it was at that certain position, or in other words at 0.33-s intervals, is obtained as the measurement results. The measurement results are shown in Fig. 13 (corresponding to the period of the time between Fig. 12a, b) and in Fig. 14 (corresponding to the period of the time between Fig. 12f, g). The horizontal axis represents time, and the vertical axis represents the change in the position of the object-tilting robot. The x-axis direction represents the horizontal (positive to the right) and the y-axis direction represents the vertical (positive downward). The point where the robot begins to tilt the object is 1 s, and the position of the robot 1 s prior to this, or in other words, 0 s, is the origin point. The change in the position in the z-axis direction (width direction from the tilting robot) at this time is considered to be small, and hence was omitted from the measurement.

Figure 13 shows that during the first tilting operation, the change in the position of the robot was within 5 mm in both the horizontal and vertical directions. In contrast, as shown in Fig. 14, during the second tilting operation, the change in both directions slightly increased. It is assumed that this difference is a result of the fact that because the pushing position of the end effector while tilting the object was set higher for the second pushing than for the first pushing, the robot tilted backwards more easily. From near the 5-s point, a change in the position of approximately 10 mm is seen in both the vertical and horizontal directions. We consider that this is a result of the fact that during the process of tilting the object, force from the object was transferred from the robot side to the two handcarts’ side so that the load of the object to the robot becomes small immediately at a certain point in time, resulting in the robot’s sag due to its looseness and softness under its own weight suddenly decreasing. Here, cares should be taken that no visible slippage existed between the robot and the object. Even under such circumstances, the object was successfully transported with the change in the position of the object-tilting robot remaining within 15 mm, and it is assumed that its tilting prevention effects are sufficient to achieve object transport.