Skip to main content
main-content
Top

Hint

Swipe to navigate through the articles of this issue

Published in: International Journal on Interactive Design and Manufacturing (IJIDeM) 2-3/2021

05-08-2021 | Original Paper

3D matching by combining CAD model and computer vision for autonomous bin picking

Authors: Le Duc Hanh, Khuong Thanh Gia Hieu

Published in: International Journal on Interactive Design and Manufacturing (IJIDeM) | Issue 2-3/2021

Login to get access
share
SHARE

Abstract

Since today, most of the manufacturing companies have the operated CAD model, so CAD-based object autonomous bin picking which using 6DOF Manipulator may be a good option that can save time and increases productivity for an assembly line. This research aims to present an effectively autonomous method that can increases productivity as well as respond quickly of changing items based on customer demand for an assembly line which using 6DOF Manipulator by combining CAD data and computer vision system. Firstly, The 3D CAD model of grasped object is projected onto six different 2D planes, then combining six views to form the final pointcloud. Secondly, a voting scheme is used to estimated the 3D pose of object which is obtained by a 3D camera. For tuning a precision of an estimation such as surface normal, angle and location of an object, Iterative closest point (ICP) algorithm is applied. Before doing experiments, the recognition algorithm is verified through the simulation program. Through implement experiments, the system proved that it is stable, good precision and applicable in production line where mass product is produced. Moreover, the developed system allows non-expert users with basic knowledge about CAD drawing and image processing can generate a pose of an object from CAD model and transmit data to a manipulator for the bin picking task.
Literature
1.
go back to reference Hardin, W.: Vision enables freestyle bin-picking, Vision System Design (2007). Hardin, W.: Vision enables freestyle bin-picking, Vision System Design (2007).
2.
go back to reference Golnabia, H., Asadpour, A.: Design and application of industrial machine vision systems. Robot. Comput. Integr. Manuf. 23(6), 630–637 (2007) CrossRef Golnabia, H., Asadpour, A.: Design and application of industrial machine vision systems. Robot. Comput. Integr. Manuf. 23(6), 630–637 (2007) CrossRef
3.
go back to reference Ulrich, M., Wiedemann, C., Steger, C.: CAD-based recognition of 3D objects in monocular images. In: Proc. IEEE International Conference on Robotics and Automation, pp. 1191–1198 (2009) Ulrich, M., Wiedemann, C., Steger, C.: CAD-based recognition of 3D objects in monocular images. In: Proc. IEEE International Conference on Robotics and Automation, pp. 1191–1198 (2009)
4.
go back to reference Casado, F., Lapido, Y.L., Losada, D. P., Alonso, A.S.: Pose estimation and object tracking using 2D image. In: 27th International Conference on Flexible Automation and Intelligent Manufacturing, 27–30 June, Modena, Italia, pp. 63–71 (2017) Casado, F., Lapido, Y.L., Losada, D. P., Alonso, A.S.: Pose estimation and object tracking using 2D image. In: 27th International Conference on Flexible Automation and Intelligent Manufacturing, 27–30 June, Modena, Italia, pp. 63–71 (2017)
5.
go back to reference Collet, A., Berenson, D., Srinivasa, S.S., Ferguson, D.: Object recognition and full pose registration from a single image for robotic manipulation. In: IEEE International Conference on Robotics and Automation, pp. 48–55 (2009) Collet, A., Berenson, D., Srinivasa, S.S., Ferguson, D.: Object recognition and full pose registration from a single image for robotic manipulation. In: IEEE International Conference on Robotics and Automation, pp. 48–55 (2009)
6.
go back to reference Ban, K., Warashina, F., Kanno, I., Kumiya, H.: Industrial intelligent robot. FANUC Tech. Rev. 16(2), 29–34 (2003) Ban, K., Warashina, F., Kanno, I., Kumiya, H.: Industrial intelligent robot. FANUC Tech. Rev. 16(2), 29–34 (2003)
7.
go back to reference Xu, J.: Rapid 3D surface profile measurement of industrial parts using two-level structured light pat-terns. Opt. Lasers. Eng 49, 907–914 (2011) CrossRef Xu, J.: Rapid 3D surface profile measurement of industrial parts using two-level structured light pat-terns. Opt. Lasers. Eng 49, 907–914 (2011) CrossRef
8.
go back to reference Bousmalis, K., Irpan, A., Wohlhart Y. P., Bai, M., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K.S., Levine, V. Vanhoucke, Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Australia, pp. 4243–4250 (2018) Bousmalis, K., Irpan, A., Wohlhart Y. P., Bai, M., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K.S., Levine, V. Vanhoucke, Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Australia, pp. 4243–4250 (2018)
9.
go back to reference Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J.A., Goldberg, K. Dex-net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In: Robotics: Science and Systems (2017) Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J.A., Goldberg, K. Dex-net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In: Robotics: Science and Systems (2017)
10.
go back to reference Tuan-Tang, L., Chyi-Yeu, L.: Bin-picking for planar objects based on a deep learning network: a case study of USB packs. Sens. Actuators 9, 3602–3618 (2019) Tuan-Tang, L., Chyi-Yeu, L.: Bin-picking for planar objects based on a deep learning network: a case study of USB packs. Sens. Actuators 9, 3602–3618 (2019)
11.
go back to reference Iglesias, A.T., Pastor-López, I., Urquijo, B.S., García-Bringas, P.: Effective bin picking approach by combining deep learning and point cloud processing techniques. In: de la Cal E.A., Villar Flecha J.R., Quintián H., Corchado E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2020. Lecture Notes in Computer Science, vol 12344. Springer, Cham Iglesias, A.T., Pastor-López, I., Urquijo, B.S., García-Bringas, P.: Effective bin picking approach by combining deep learning and point cloud processing techniques. In: de la Cal E.A., Villar Flecha J.R., Quintián H., Corchado E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2020. Lecture Notes in Computer Science, vol 12344. Springer, Cham
12.
go back to reference Benjamin, J., Tevon, W., Remi, G., Konrad, A.: Pose estimation and bin picking for deformable products. IFAC-PapersOnLine 52(30), 361–366 (2019) CrossRef Benjamin, J., Tevon, W., Remi, G., Konrad, A.: Pose estimation and bin picking for deformable products. IFAC-PapersOnLine 52(30), 361–366 (2019) CrossRef
13.
go back to reference Chun-Tse, L., Cheng-Han, T., Jen-Yuan, C.: A CAD-free random bin picking system for fast changeover on multiple objects. In: ASME 2020 29th Conference on Information Storage and Processing Systems (2020) Chun-Tse, L., Cheng-Han, T., Jen-Yuan, C.: A CAD-free random bin picking system for fast changeover on multiple objects. In: ASME 2020 29th Conference on Information Storage and Processing Systems (2020)
14.
go back to reference Chen, J., Fujinami, T., Li, E.: Deep bin picking with reinforcement learning. In: Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018, pp. 1–8 Chen, J., Fujinami, T., Li, E.: Deep bin picking with reinforcement learning. In: Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018, pp. 1–8
15.
go back to reference Wu, G., Bin, L., Qibing, Z., Huang, M., Guo, Y.: Using color and 3D geometry features to segment fruit point cloud and improve fruit recognition accuracy. Comput. Electron. Agric. 174, 1–8 (2020) Wu, G., Bin, L., Qibing, Z., Huang, M., Guo, Y.: Using color and 3D geometry features to segment fruit point cloud and improve fruit recognition accuracy. Comput. Electron. Agric. 174, 1–8 (2020)
16.
go back to reference Aldoma, A. et al. CAD-model recognition and 6DOF pose estimation using 3D cues. In: Proc. IEEE Int. Conf. Comput. Vis. pp. 585–592 (2011) Aldoma, A. et al. CAD-model recognition and 6DOF pose estimation using 3D cues. In: Proc. IEEE Int. Conf. Comput. Vis. pp. 585–592 (2011)
17.
go back to reference Song, K.T., Wu, C.H., Jiang, S.Y.: CAD-based pose estimation design for random bin picking using a RGB-D camera. J. Intell. Robot. Syst. 87, 455–470 (2017) CrossRef Song, K.T., Wu, C.H., Jiang, S.Y.: CAD-based pose estimation design for random bin picking using a RGB-D camera. J. Intell. Robot. Syst. 87, 455–470 (2017) CrossRef
18.
go back to reference Li, M., Hashimoto, K.: Fast and robust pose estimation algorithm for bin picking using point pair feature. In: 24th International Conference on Pattern Recognition (ICPR), pp. 1604–1609 (2018) Li, M., Hashimoto, K.: Fast and robust pose estimation algorithm for bin picking using point pair feature. In: 24th International Conference on Pattern Recognition (ICPR), pp. 1604–1609 (2018)
19.
go back to reference Hanh, L.D., Tu, H.B.: Computer vision for industrial robot in planar bin picking application. Adv. Sci. Technol. Eng. Syst. J. 5(6), 1244–1249 (2020) CrossRef Hanh, L.D., Tu, H.B.: Computer vision for industrial robot in planar bin picking application. Adv. Sci. Technol. Eng. Syst. J. 5(6), 1244–1249 (2020) CrossRef
22.
23.
go back to reference Hallad, S.A., Banapurmath, N.R., Patil, A.Y., Hunashyal, A.M., Shettar, A.S.: Studies on the effect of multi-walled carbon nanotube–reinforced polymer-based nano-composites using finite element analysis software tool. Proc. Inst. Mech. Eng. Part N J. Nanomater. Nanoeng. Nanosyst. 230(4), 200–212 (2016) Hallad, S.A., Banapurmath, N.R., Patil, A.Y., Hunashyal, A.M., Shettar, A.S.: Studies on the effect of multi-walled carbon nanotube–reinforced polymer-based nano-composites using finite element analysis software tool. Proc. Inst. Mech. Eng. Part N J. Nanomater. Nanoeng. Nanosyst. 230(4), 200–212 (2016)
24.
go back to reference Vishalagoud, S.P., Farheen, B., Kurahatti, R.V., Arun, Y.P., Raju, G.U., Afzal, A., Manzoore, E., Soudagar, M., Ravinder Kumar, C., Ahamed, S.: A study of sound pressure level (SPL) inside the truck cabin for new acoustic materials: an experimental and FEA approach. Alex Eng J 60(6), 5949–5976 (2021) CrossRef Vishalagoud, S.P., Farheen, B., Kurahatti, R.V., Arun, Y.P., Raju, G.U., Afzal, A., Manzoore, E., Soudagar, M., Ravinder Kumar, C., Ahamed, S.: A study of sound pressure level (SPL) inside the truck cabin for new acoustic materials: an experimental and FEA approach. Alex Eng J 60(6), 5949–5976 (2021) CrossRef
26.
go back to reference Hallad, S.A., Patil, A.Y., BanapurmathNR, H.A.M., Shettar, A.S., Ayachit, N.H.: Experimental and numerical validation on the utilization of polymer based nano-composites for structural applications using FEA software tool. Mater. Focus 6(6), 685–690 (2017) CrossRef Hallad, S.A., Patil, A.Y., BanapurmathNR, H.A.M., Shettar, A.S., Ayachit, N.H.: Experimental and numerical validation on the utilization of polymer based nano-composites for structural applications using FEA software tool. Mater. Focus 6(6), 685–690 (2017) CrossRef
27.
go back to reference Katz, S., Tal, A., Basri, R.: Direct visibility of point sets. In: Proceeding ACM SIGGRAPH Conference on Computer Graphic, pp. 1–8 (2007) Katz, S., Tal, A., Basri, R.: Direct visibility of point sets. In: Proceeding ACM SIGGRAPH Conference on Computer Graphic, pp. 1–8 (2007)
28.
go back to reference Hanh, L.D., Duc, L.M.: Planar object recognition for bin picking application. In: Proceedings of 5th NAFOSTED Conference on Information and Computer Science (NICS), Ho Chi Minh, pp. 213–218 (2018) Hanh, L.D., Duc, L.M.: Planar object recognition for bin picking application. In: Proceedings of 5th NAFOSTED Conference on Information and Computer Science (NICS), Ho Chi Minh, pp. 213–218 (2018)
29.
go back to reference Rusu, R.B.: Semantic 3D object maps for everyday manipulation in human living environments. Künstl Intell. 24, 345–348 (2010) CrossRef Rusu, R.B.: Semantic 3D object maps for everyday manipulation in human living environments. Künstl Intell. 24, 345–348 (2010) CrossRef
30.
go back to reference Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition, pp. 998–1005 (2010) Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition, pp. 998–1005 (2010)
32.
go back to reference Kverneland, K. O.: The ISO System of Limits and Fits—Tolerances and Deviations. (2007) Kverneland, K. O.: The ISO System of Limits and Fits—Tolerances and Deviations. (2007)
Metadata
Title
3D matching by combining CAD model and computer vision for autonomous bin picking
Authors
Le Duc Hanh
Khuong Thanh Gia Hieu
Publication date
05-08-2021
Publisher
Springer Paris
Published in
International Journal on Interactive Design and Manufacturing (IJIDeM) / Issue 2-3/2021
Print ISSN: 1955-2513
Electronic ISSN: 1955-2505
DOI
https://doi.org/10.1007/s12008-021-00762-4

Other articles of this Issue 2-3/2021

International Journal on Interactive Design and Manufacturing (IJIDeM) 2-3/2021 Go to the issue

Premium Partner