Top

The International Journal of Advanced Manufacturing Technology

Published in:

06-06-2019 | ORIGINAL ARTICLE

Towards seamless human robot collaboration: integrating multimodal interaction

Authors: Stergios Papanastasiou, Niki Kousi, Panagiotis Karagiannis, Christos Gkournelos, Apostolis Papavasileiou, Konstantinos Dimoulas, Konstantinos Baris, Spyridon Koukas, George Michalos, Sotiris Makris

Published in: The International Journal of Advanced Manufacturing Technology | Issue 9/2019

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This paper discusses the challenges in the collaboration between human operators and industrial robots for assembly operations focusing on safety and simplified interaction. A case study is presented, involving perception technologies for the robot in conjunction with wearable devices used by the operator. In terms of robot perception, a manual guidance module, an air pressor contact sensor namely skin, and a vision system for recognition and tracking of objects have been developed and integrated. Concerning the wearable devices, an advanced user interface including audio and haptic commands accompanied by augmented reality technology are used to support the operator and provide awareness by visualizing information related to production and safety aspects. In parallel, safety functionalities are implemented through collision detection technologies such as a safety skin and safety monitored regions delimiting the area of the robot activities. The complete system is coordinated under a common integration platform and it is validated in a case study of the white goods industry.

previous article A fuzzy bi-objective flexible cell scheduling optimization model under green and energy-efficient strategy using Pareto-based algorithms: SATPSPGA, SANRGA, and NSGA-II

next article Augmented reality application to support the assembly of highly customized products and to adapt to production re-scheduling

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Tsarouchi P, Makris S, Chryssolouris G (2016) Human-robot interaction review and challenges on task planning and programming. Int J Comput Integr Manuf 29:916–931. https://doi.org/10.1080/0951192X.2015.1130251 CrossRef

Chryssolouris G (2006) Manufacturing systems: theory and practice, 2nd edn. Springer-Verlag, New York

Bortolini M, Faccio M, Gamberi M, Pilati F (2018) Motion analysis system (MAS) for production and ergonomics assessment in the manufacturing processes. Comput Ind Eng. https://doi.org/10.1016/j.cie.2018.10.046

Bortolini M, Faccio M, Gamberi M, Pilati F (2017a) Multi-objective assembly line balancing considering component picking and ergonomic risk. Comput Ind Eng 112:348–367. https://doi.org/10.1016/j.cie.2017.08.029 CrossRef

Bortolini M, Ferrari E, Gamberi M, Pilati F, Faccio M (2017b) Assembly system design in the industry 4.0 era: a general framework. 20th IFAC World Congress. IFAC-PapersOnLine 50:5700–5705. https://doi.org/10.1016/j.ifacol.2017.08.1121 CrossRef

Kousi N, Michalos G, Aivaliots S, Makris S (2018) An outlook on future assembly systems introducing robotic mobile dual arm workers. 51st CIRP Conference on Manufacturing System. Proced CIRP 72:33–38. https://doi.org/10.1016/j.procir.2018.03.130 CrossRef

Chryssolouris G, Mourtzis D (2012) Proceedings on manufacturing, modeling, management and control. 45th CIRP Conference on Manufacturing Systems, Procedia CIRP 3:1–650

Pilz GmbH (2015) Safe camera system SafetyEYE. https://www.pilz.com/en-DE/eshop/00014000337042/SafetyEYE-Safe-camera-system. Accessed 2 Dec 2015

Bley H, Reinhart G, Seliger G, Bernardi M, Korne T (2004) Appropriate human involvement in assembly and disassembly. CIRP Ann 53:487–509. https://doi.org/10.1016/S0007-8506(07)60026-2 CrossRef

10.

Kruger J, Bernhardt R, Surdilovic D, Spur G (2006) Intelligent assist systems for flexible assembly. CIRP Ann 55:29–33. https://doi.org/10.1016/S0007-8506(07)60359-X CrossRef

11.

Helms E, Schraft RD, Hagele M (2002) Rob@work: robot assistant in industrial environments. Proceedings of the 11th IEEE Int. Workshop on Robot and Human Interactive Communication, pp 399–404. https://doi.org/10.1109/ROMAN.2002.1045655

12.

Wannasuphoprasit W, Akella P, Peshkin M, Colgate JE (1998) A novel material handling technology. International Mechanical Engineering Congress and Exposition, pp 1–7

13.

Brecher C, Schroter B, Almeida C (2005) Development and programming of portable robot systems for material handling tasks. Proceedings of the CIRP International Conference on Reconfigurable Manufacturing

14.

Bernhardt R, Surdilovic D, Katschinski V, Schroer K, Schroer K (2008) Next generation of flexible assembly systems. Innovation in Manufacturing Networks, pp 279–288. https://doi.org/10.1007/978-0-387-09492-2_30

15.

Bernhardt R, Surdilovic D, Katschinski V, Schroer K (2008) Flexible assembly systems through human integration. IEEE SMC International Conference on Distributed Human-Machine Systems Proceedings, pp 497–502. https://doi.org/10.3182/20070523-3-ES-4908.00041 CrossRef

16.

Michalos G, Makris S, Tsarouchi P, Guasch T, Kontovrakis D, Chryssolouris G (2015) Design considerations for safe human–robot collaborative workplaces. Proced CIRP 37:248–253. https://doi.org/10.1016/j.procir.2015.08.014 CrossRef

17.

Michalos G, Makris S, Spiliotopoulos J, Misios I, Tsarouchi P, Chryssolouris G (2014) ROBO-PARTNER: seamless human-robot cooperation for intelligent, flexible and safe operations in the assembly factories of the future. Proced CIRP 23:71–76. https://doi.org/10.1016/j.procir.2014.10.079 CrossRef

18.

Goodrich M, Schultz C (2008) Human-robot interaction: a survey. Found Trends Hum Comput Interact 1(3):203–275. https://doi.org/10.1561/1100000005 CrossRefMATH

19.

Universal robots https://www.universal-robots.com/. Accessed on 2019

20.

KUKA robots https://www.kuka.com/. Accessed on 2019

21.

Michalos G, Kousi N, Karagiannis P, Gkournelos C, Dimoulas K, Koukas S, Mparis K, Papavasiliou A, Makris S (2018) Seamless human robot collaborative assembly—an automotive case study. Mechatronics 55:194–211. https://doi.org/10.1016/j.mechatronics.2018.08.006 CrossRef

22.

Elkeman N Tactile sensors. Fraunhofer IFF. http://www.iff.fraunhofer.de/en/business-units/robotic-systems/technologies/tactile-sensor-systems.html. Accessed on 2016

23.

Perez L, Rodriguez I, Rodriguez N, Usamentiaga R, Garcia DF (2016) Robot guidance using machine vision techniques in industrial environments: a comparative review. Sensors 16(3):335. https://doi.org/10.3390/s16030335 CrossRef

24.

Statista (2018) Household appliances. Statista digital market outlook. https://www.statista.com/outlook/256/102/home-appliances/europe#. Accessed on 2018

25.

Kang MK, Lee S, Kim JH (2014) Shape optimization of a mechanically decoupled six-axis force/torque sensor. Sensors Actuators 209:41–51. https://doi.org/10.1016/j.sna.2014.01.001 CrossRef

26.

Chen S, Li Y, Kwok NM (2011) Active vision in robotic systems: a survey of recent developments. Int J Robot Res 30:1343–1377. https://doi.org/10.1177/0278364911410755 CrossRef

27.

Davies ER (1998) Automated visual inspection. Mach Vis 19:471–502. https://doi.org/10.1016/B978-0-12-206090-8.50027-X CrossRef

28.

Sanz J, Petkovic D (1988) Machine vision algorithm for automated inspection of thin-film disk heads. IEEE Trans PAMI 10:830–848. https://doi.org/10.1109/34.9106 CrossRef

29.

Tucker JW (1989) Inside beverage can inspection: an application from start to finish. Proceedings of the Vision '89 Conference

30.

Ker J, Kengskool K (1990) An efficient method for inspecting machine parts by a fixtureless machine vision system. Proceedings of the Vision '90 Conference

31.

Li H, Lin JC (1994) Using fuzzy logic to detect dimple defects of polished wafer surfaces. IEEE Trans Ind Appl 30:1530–1543. https://doi.org/10.1109/28.287528 CrossRef

32.

Rentzos L, Papanastasiou S, Papakostas N, Chryssolouris G (2013) Augmented reality for human-based assembly: using product and process semantics. IFAC Proceed 46(15):98–101. https://doi.org/10.3182/20130811-5-US-2037.00053 CrossRef

33.

Michalos G, Karagiannis P, Makris S, Tokcalar O, Chryssolouris G (2015) Augmented reality (AR) applications for supporting human-robot interactive cooperation. Proced CIRP 41:370–375. https://doi.org/10.1016/j.procir.2015.12.005 CrossRef

34.

Makris S, Karagiannis P, Koukas S, Matthaiakis A-S (2016) Augmented reality system for operator support in human–robot collaborative assembly. CIRP Ann 65:61–64. https://doi.org/10.1016/j.cirp.2016.04.038 CrossRef

35.

Gkournelos C, Karagiannis P, Kousi N, Michalos G, Koukas S, Makris S (2018) Application of wearable devices for supporting operators in human-robot cooperative assembly tasks. 7th CIRP Conference on Assembly Technologies And Systems, (CATS 2018), Tianjin, China

36.

Kokkalis K, Michalos G, Aivaliotis P, Makris S (2018) An approach for implementing power and force limiting in sensorless industrial robots. 7th CIRP Conference on Assembly Technologies And Systems, (CATS 2018), Tianjin, China

37.

Blue Danube Robotics http://www.bluedanuberobotics.com. Accessed on 2016

38.

Argyrou A, Giannoulis C, Sardelis A, Karagiannis P, Michalos G, Makris S (2018) A data fusion system for controlling the execution status in human-robot collaborative cells. Proced CIRP 76:193–198. https://doi.org/10.1016/j.procir.2018.01.012 CrossRef

39.

Karagiannis P, Giannoulis C, Michalos G, Makris S (2018) Configuration and control approach for flexible production stations. Proced CIRP 78:166–171. https://doi.org/10.1016/j.procir.2018.09.053 CrossRef

40.

Romanelli F (2011) Advanced methods for robot-environment interaction towards an industrial robot aware of its volume. J Robot 2011:1–12. https://doi.org/10.1155/2011/389158 CrossRef

41.

Makris S, Rentzos L, Pintzos G, Mavrikios D, Chryssolouris G (2012) Semantic-based taxonomy for immersive product design using VR techniques. CIRP Ann 61:147–150. https://doi.org/10.1016/j.cirp.2012.03.008 CrossRef

42.

Patil M, Joshi N, Tilekar A, Shinde P (2015) Li-fi based voice controlled robot. National Conference on Emerging Trends in Advanced Communication Technologies 35–37

43.

Silaghi H, Rohde U, Spoial V, Silaghi A, Gergely E, Nagy Z (2014) Voice command of an industrial robot in a noisy environment. Fundamentals of Electrical Engineering (ISFEE), pp 1–5. https://doi.org/10.1109/ISFEE.2014.7050596

Title: Towards seamless human robot collaboration: integrating multimodal interaction
Authors: Stergios Papanastasiou
Niki Kousi
Panagiotis Karagiannis
Christos Gkournelos
Apostolis Papavasileiou
Konstantinos Dimoulas
Konstantinos Baris
Spyridon Koukas
George Michalos
Sotiris Makris
Publication date: 06-06-2019
Publisher: Springer London
Published in: The International Journal of Advanced Manufacturing Technology / Issue 9/2019
Print ISSN: 0268-3768
Electronic ISSN: 1433-3015
DOI: https://doi.org/10.1007/s00170-019-03790-3

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 9/2019

Anomaly detection in Skin Model Shapes using machine learning classifiers

Functional verification of distributed automation systems

CoMM: a consensus algorithm for multi-agent-based manufacturing system to deal with perturbation

Current and future Industry 4.0 capabilities for information and knowledge sharing

Intelligent configuring for agile joint jig based on smart composite jig model

Production logistics and human-computer interaction—state-of-the-art, challenges and requirements for the future

Premium Partners