Top

Published in:

2023 | OriginalPaper | Chapter

33. Convergence of IoT and Augmented Reality

Authors : Alexis Morris, Georgina Yeboah

Published in: Springer Handbook of Augmented Reality

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This chapter contends that the Internet of Things (IoT), which consists of technologies for networked embedded devices and decentralized software applications, must also have rich interface solutions that both inform and engage its users. This is a new frontier in IoT human-computer interaction (HCI), and meeting these challenges requires interface designs that present information to users in a context-sensitive manner, which can inform users while allowing them to remain engaged in their environmental tasks. Mixed reality, combined with context sensing, also known as X-Reality (XR), allows the display of 3D content in situ, providing novel interface design possibilities for IoT edge devices. These XR-IoT (XRI) hybrid systems can become more personal, immersive, embedded, information-rich, decentralized, multiuser, and agent-driven. This repositions the IoT from the passive background embedded environment into active, engaging, foreground information infrastructures. This chapter highlights the challenges and considerations to meet the convergence of the IoT with mixed reality, toward a transformation of IoT devices into hybrid virtual and physical objects that adapt to their user’s context, and presents information in engaging ways. Such a merger will contribute to the IoT broadly and impact HCI across existing and future IoT deployments.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter The Augmented Reality Internet of Things: Opportunities of Embodied Interactions in Transreality

next chapter Digital Twin and Extended Reality: Strategic Approach and Practical Implementation

Ahmed, E., Yaqoob, I., Gani, A., Imran, M., Guizani, M.: Internet-of-things-based smart environments: State of the art, taxonomy, and open research challenges. IEEE Wirel. Commun. 23, 10–16 (2016). https://doi.org/10.1109/MWC.2016.7721736CrossRef

Bernal, G., Colombo, S., Al Ai Baky, M., Casalegno, F.: Safety++ designing IoT and wearable systems for industrial safety through a user centered design approach. In: Proceedings of the 10th International Conference on Pervasive Technologies Related to Assistive Environments, pp. 163–170 (2017)

Bilda, Z., Edmonds, E., Candy, L.: Designing for creative engagement. Design Stud. 29, 525–540 (2007). https://doi.org/10.1016/j.destud.2008.07.009CrossRef

Billinghurst, M., Kato, H., Poupyrev, I., et al.: Tangible augmented reality. ACM SIGGRAPH Asia 7(2), 1–10 (2008)

Billinghurst, M., Clark, A., Lee, G., et al.: A survey of augmented reality. Found. Trends® Hum.-Comput. Interact. 8(2-3), 73–272 (2015)

Blanco-Novoa, Ó., Fraga-Lamas, P., Vilar-Montesinos, M.A., Fernández-Caramés, T.M.: Towards the internet of augmented things: an open-source framework to interconnect IoT devices and augmented reality systems. In: Multidisciplinary Digital Publishing Institute Proceedings, vol. 42, p. 50 (2019)

Bogost, I.: Alien Phenomenology, or What It’s Like to Be a Thing. University of Minnesota (2012)

Bridle, J.: New dark age: Technology and the end of the future. Verso Books (2018)

Brudy, F., Holz, C., Rädle, R., Wu, C.J., Houben, S., Klokmose, C.N., Marquardt, N.: Cross-device taxonomy: survey, opportunities and challenges of interactions spanning across multiple devices. In: Conference on Human Factors in Computing Systems - Proceedings pp. 1–28 (2019). https://doi.org/10.1145/3290605.3300792

10.

Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., Ebel, P.: The future of human-ai collaboration: A taxonomy of design knowledge for hybrid intelligence systems. In: Proceedings of the 52nd Hawaii International Conference on System Sciences, vol. 6, pp. 274–283 (2019). https://doi.org/10.24251/hicss.2019.034

11.

Dey, A.K., Abowd, G.D.: The what, who, where, when, why and how of context-awareness (2000)

12.

Di Geronimo, L., Braz, L., Fregnan, E., Palomba, F., Bacchelli, A.: Ui dark patterns and where to find them: a study on mobile applications and user perception. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2020)

13.

Fisher, J.A.: Utilizing the mixed reality cube taxonomy for interactive documentary research. AltMM 2016 - Proceedings of the 1st International Workshop on Multimedia Alternate Realities, co-located with ACM Multimedia 2016 pp. 9–14 (2016). https://doi.org/10.1145/2983298.2983299

14.

Flavián, C., Ibáñez-Sánchez, S., Orús, C.: The impact of virtual, augmented and mixed reality technologies on the customer experience. J. Bus. Res. 100, 547–560 (2019)CrossRef

15.

Gubbi, J., Buyya, R., Marusic, S., Palaniswami, M.: Internet of Things (IoT): A vision, architectural elements, and future directions. Fut. Gener. Comput. Syst. 29(7), 1645–1660 (2013)CrossRef

16.

Gushima, K., Nakajima, T.: A design space for virtuality-introduced Internet of Things. Fut. Internet 9(4), 60 (2017)CrossRef

17.

Harman, G.: Heidegger Explained, vol. 4. Open Court Publishing (2007)

18.

Iatsyshyn, V.A., Kovach, O.V., Romanenko, O.Y., Deinega, I.I., Iatsyshyn, V.A., Popov, O.O., Kutsan, G.Y., Artemchuck, O.V., Burov, Y.O., Lytvynova, H.S.: Application of augmented reality technologies for preparation of specialists of new technological era. English Article 2018, 181–200 (2020)

19.

Jeschke, S., Brecher, C., Meisen, T., Özdemir, D., Eschert, T.: Industrial internet of things and cyber manufacturing systems. In: Industrial Internet of Things, pp. 3–19. Springer, Berlin (2017)

20.

Jing, Q., Vasilakos, A.V., Wan, J., Lu, J., Qiu, D.: Security of the Internet of Things: perspectives and challenges. Wirel. Netw. 20(8), 2481–2501 (2014)CrossRef

21.

Jo, D., Kim, G.J.: Ariot: scalable augmented reality framework for interacting with internet of things appliances everywhere. IEEE Trans. Consum. Electron. 62(3), 334–340 (2016)CrossRef

22.

Jo, D., Kim, G.J.: In-situ ar manuals for IoT appliances. In: 2016 IEEE International Conference on Consumer Electronics (ICCE), pp. 409–410. IEEE, Piscataway (2016)

23.

Jo, D., Kim, G.J.: Ar enabled IoT for a smart and interactive environment: a survey and future directions. Sensors 19(19), 4330 (2019)CrossRef

24.

Jo, D., Kim, G.J.: IoT+ AR: pervasive and augmented environments for “digi-log” shopping experience. Hum.-Centric Comput. Inform. Sci. 9(1), 1–17 (2019)

25.

Jo, D., Kim, K.H., Kim, G.J.: Effects of avatar and background representation forms to co-presence in mixed reality (MR) tele-conference systems. In: SIGGRAPH ASIA 2016 Virtual Reality Meets Physical Reality: Modelling and Simulating Virtual Humans and Environments, pp. 1–4 (2016)

26.

Karthick, M., Kumar, C.S., Robert, T.P.: Investigation of Human Factors Using HFACS Framework—A Case Study for Unintended Reactor Trip Events in NPP. Springer, Singapore (2020). https://doi.org/10.1007/978-981-13-9008-1CrossRef

27.

Kayes, A.S., Kalaria, R., Sarker, I.H., Islam, M.S., Watters, P.A., Ng, A., Hammoudeh, M., Badsha, S., Kumara, I.: A survey of context-aware access control mechanisms for cloud and fog networks: Taxonomy and open research issues. Sensors (Switzerland) 20 (2020). https://doi.org/10.3390/s20092464

28.

Hoon Kim, T., Ramos, C., Mohammed, S.: Smart city and IoT. Fut. Gener. Comput. Syst. 76, 159–162 (2017). https://doi.org/10.1016/j.future.2017.03.034CrossRef

29.

Kim, Y.M., Rhiu, I., Yun, M.H.: A systematic review of a virtual reality system from the perspective of user experience. Int. J. Hum.-Comput. Interact. 36, 893–910 (2020). https://doi.org/10.1080/10447318.2019.1699746CrossRef

30.

Kranz, M., Holleis, P., Schmidt, A.: Embedded interaction: interacting with the Internet of Things. IEEE Internet Comput. 14(2), 46–53 (2009)CrossRef

31.

Kress, B., Saeedi, E., Brac-de-la Perriere, V.: The segmentation of the HMD market: optics for smart glasses, smart eyewear, AR and VR headsets. In: Photonics Applications for Aviation, Aerospace, Commercial, and Harsh Environments V, vol. 9202, p. 92020D. International Society for Optics and Photonics (2014)

32.

Kress, B.C., Cummings, W.J.: 11-1: Invited paper: Towards the ultimate mixed reality experience: Hololens display architecture choices. In: SID Symposium Digest of Technical Papers, vol. 48, pp. 127–131. Wiley Online Library (2017)

33.

Lee, L.H., Hui, P.: Interaction methods for smart glasses: a survey. IEEE Access 6, 28712–28732 (2018)CrossRef

34.

Leotta, F., Mecella, M., Sora, D., Catarci, T.: Surveying human habit modeling and mining techniques in smart spaces. Fut. Internet 11 (2019). https://doi.org/10.3390/fi11010023

35.

Lupiana, D., Driscoll, C.O., Mtenzi, F.: Defining smart space in the context of ubiquitous computing. In: Special Issue onICIT 2009 Conference -Web and Agent Systems, vol. 4, pp. 516–524 (2009)

36.

Mann, S., Furness, T., Yuan, Y., Iorio, J., Wang, Z.: All reality: Values, taxonomy, and continuum, for virtual, augmented, extended/mixed (x), mediated (x,y), and multimediated reality/intelligence. Awe (2018)

37.

Mignonneau, L., Sommerer, C.: Designing emotional, metaphoric, natural and intuitive interfaces for interactive art, edutainment and mobile communications. Comput. Graph. (Pergamon) 29, 837–851 (2005). https://doi.org/10.1016/j.cag.2005.09.001

38.

Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inform. Syst. 77(12), 1321–1329 (1994)

39.

Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: a class of displays on the reality-virtuality continuum. In: Telemanipulator and Telepresence Technologies, vol. 2351, pp. 282–292. International Society for Optics and Photonics (1995)

40.

Milhorat, P., Schlögl, S., Chollet, G., Boudy, J., Esposito, A., Pelosi, G.: Building the next generation of personal digital assistants. In: 2014 1st International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 458–463. IEEE, Piscataway (2014)

41.

Morris, A., Lessio, N.: Deriving privacy and security considerations for core: an indoor IoT adaptive context environment. In: Proceedings of the ACM Conference on Computer and Communications Security pp. 2–11 (2018). https://doi.org/10.1145/3267357.3267363

42.

Morris, A., Guan, J., Lessio, N., Shao, Y.: Toward mixed reality hybrid objects with IoT avatar agents. In: 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 766–773. IEEE, Piscataway (2020)

43.

Muhanna, M.A.: Virtual reality and the cave: Taxonomy, interaction challenges and research directions. J. King Saud Univ. Comput. Inform. Sci. 27, 344–361 (2015). https://doi.org/10.1016/j.jksuci.2014.03.023

44.

Nielsen, J.: Iterative user-interface design. Computer 26(11), 32–41 (1993)CrossRef

45.

Norman, D.: The Design of Everyday Things, doubleday/edn. Basic Books (1990)

46.

Nuamah, J., Seong, Y.: Human machine interface in the internet of things (IoT). In: 2017 12th System of Systems Engineering Conference (SoSE), pp. 1–6. IEEE, Piscataway (2017)

47.

Paradiso, J.A., Landay, J.A.: Guest editors’ introduction: cross-reality environments. IEEE Pervasive Comput. 8(3), 14–15 (2009)CrossRef

48.

Perera, C., Zaslavsky, A., Christen, P., Georgakopoulos, D.: Context aware computing for the internet of things: a survey. IEEE Commun. Surv. Tutorials 16, 414–454 (2014). https://doi.org/10.1109/SURV.2013.042313.00197CrossRef

49.

Phupattanasilp, P., Tong, S.R.: Augmented reality in the integrative internet of things (AR-IoT): Application for precision farming. Sustainability 11(9), 2658 (2019)CrossRef

50.

Rokhsaritalemi, S., Sadeghi-Niaraki, A., Choi, S.M.: A review on mixed reality: current trends, challenges and prospects. Appl. Sci. (Switzerland) 10 (2020). https://doi.org/10.3390/app10020636

51.

Shao, Y., Lessio, N., Morris, A.: IoT avatars: mixed reality hybrid objects for core ambient intelligent environments. Proc. Comput. Sci. 155, 433–440 (2019). https://doi.org/10.1016/j.procs.2019.08.060CrossRef

52.

Shin, D.: A socio-technical framework for internet-of-things design: a human-centered design for the Internet of Things. Telematics Inform. 31(4), 519–531 (2014)CrossRef

53.

Shinde, G.R., Dhotre, P.S., Mahalle, P.N., Dey, N.: Internet of Things Integrated Augmented Reality. Springer, Berlin (2020)

54.

Shirehjini, A.A.N., Semsar, A.: Human interaction with IoT-based smart environments. Multimedia Tools Appl. 76(11), 13343–13365 (2017)CrossRef

55.

Thierer, A., Castillo, A.: Projecting the growth and economic impact of the Internet of Things. George Mason University, Mercatus Center, vol. 15 (2015)

56.

Truelove, A., Chowdhury, F.N., Gnawali, O., Alipour, M.A.: Topics of concern: identifying user issues in reviews of IoT apps and devices. In: 2019 IEEE/ACM 1st International Workshop on Software Engineering Research & Practices for the Internet of Things (SERP4IoT), pp. 33–40. IEEE, Piscataway (2019)

57.

Tuladhar, S., AlSallami, N., Alsadoon, A., Prasad, P.W., Alsadoon, O.H., Haddad, S., Alrubaie, A.: A recent review and a taxonomy for hard and soft tissue visualization-based mixed reality. Int. J. Med. Robot. 16(5), 1–22 (2020). https://doi.org/10.1002/rcs.2120CrossRef

58.

Ungureanu, D., Bogo, F., Galliani, S., Sama, P., Duan, X., Meekhof, C., Stühmer, J., Cashman, T.J., Tekin, B., Schönberger, J.L., et al.: Hololens 2 research mode as a tool for computer vision research (2020). arXiv preprint arXiv:2008.11239

59.

Vicente, K.: The human factor: revolutionizing the way we live with technology. Vintage Canada (2004)

60.

West, D.M.: How 5g technology enables the health internet of things. Brookings Center for Technology Innovation, vol. 3, pp. 1–20 (2016)

61.

White, G., Cabrera, C., Palade, A., Clarke, S.: Augmented reality in IoT. In: International Conference on Service-Oriented Computing, pp. 149–160. Springer, Berlin (2018)

62.

Zemmoudj, S., Bermad, N., Omar, M.: Context-aware pseudonymization and authorization model for IoT-based smart hospitals. J. Ambient Intell. Hum. Comput. 10, 4473–4490 (2019). https://doi.org/10.1007/s12652-018-1129-0CrossRef

63.

Zollmann, S., Grasset, R., Langlotz, T., Lo, W.H., Mori, S., Regenbrecht, H.: Visualization techniques in augmented reality: a taxonomy, methods and patterns. IEEE Trans. Vis. Comput. Graph. 2626, 1–20 (2020). https://doi.org/10.1109/TVCG.2020.2986247

Title: Convergence of IoT and Augmented Reality
Authors: Alexis Morris
Georgina Yeboah
Publisher: Springer International Publishing
Book: Springer Handbook of Augmented Reality
Print ISBN: 978-3-030-67821-0

Electronic ISBN: 978-3-030-67822-7

Copyright Year: 2023
DOI: https://doi.org/10.1007/978-3-030-67822-7_33

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner