Top

Published in:

2022 | OriginalPaper | Chapter

Unified Model and Framework for Interactive Mixed Entity Systems

Authors : Guillaume Bataille, Valérie Gouranton, Jérémy Lacoche, Danielle Pelé, Bruno Arnaldi

Published in: Computer Vision, Imaging and Computer Graphics Theory and Applications

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Mixed reality, natural user interfaces, and the internet of things converge towards an advanced sort of interactive system. These systems enable new forms of interactivity, allowing intuitive user interactions with ubiquitous services in mixed environments. However, they require to synchronize multiple platforms and various technologies. Their heterogeneity makes them complex, and sparsely interoperable or extensible. Therefore, designers and developers require new models, tools, and methodologies to support their creation. We present a unified model of the entities composing these systems, breaking them down into graphs of mixed entities. This model decorrelates real and virtual but still describes their interplay. It characterizes and classifies both the external and internal interactions of mixed entities. We also present a design and implementation framework based on our unified model. Our framework takes advantage of our model to simplify, accelerate, and unify the production of these systems. We showcase the use of our framework by designers and developers in the case of a smart building management system.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Correction to: Regression-Based 3D Hand Pose Estimation for Human-Robot Interaction

next chapter Skeleton-and-Trackball Interactive Rotation Specification for 3D Scenes

https://z-wavealliance.org/.

https://developers.google.com/ar/.

https://developer.apple.com/augmented-reality/arkit/.

https://github.com/Microsoft/MixedRealityToolkit-Unity.

https://unity.com/fr.

https://github.com/genielabs/zwave-lib-dotnet.

https://lora-alliance.org/.

https://developer.vuforia.com/.

https://github.com/artoolkit.

Alce, G., Roszko, M., Edlund, H., Olsson, S., Svedberg, J., Wallergård, M.: [POSTER] AR as a user interface for the internet of things—comparing three interaction models. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 81–86, October 2017. https://doi.org/10.1109/ISMAR-Adjunct.2017.37

Atzori, L., Iera, A., Morabito, G.: The internet of things: a survey. Comput. Netw. 54(15), 2787–2805 (2010). https://doi.org/10.1016/j.comnet.2010.05.010. http://dx.doi.org/10.1016/j.comnet.2010.05.010

Bataille, G., Gouranton, V., Lacoche, J., Pelé, D., Arnaldi, B.: A unified design & development framework for mixed interactive systems. In: VISIGRAPP, GRAPP, the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Valetta, Malta, pp. 49–60 (2020)

Blackler, A., Hurtienne, J.: Towards a unified view of intuitive interaction: definitions, models and tools across the world. MMI-Interaktiv 13 (2007)

Blake, J.: The natural user interface revolution. In: Natural User Interfaces in .Net, pp. 1–43. Manning publications edn. (2012)

Bouville, R., Gouranton, V., Boggini, T., Nouviale, F., Arnaldi, B.: #FIVE: high-level components for developing collaborative and interactive virtual environments. In: 2015 IEEE 8th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS), pp. 33–40. IEEE, Arles, March 2015. https://doi.org/10.1109/SEARIS.2015.7854099. http://ieeexplore.ieee.org/document/7854099/

Bouzekri, E., Canny, A., Martinie, C., Palanque, P.: A generic software and hardware architecture for hybrid interactive systems. In: EICS 2018 - Workshop on Heterogeneous Models and Modeling Approaches for Engineering of Interactive Systems, Paris, France, June 2018

Claude, G., Gouranton, V., Arnaldi, B.: Versatile scenario guidance for collaborative virtual environments. In: Proceedings of 10th International Conference on Computer Graphics Theory and Applications (GRAPP 2015), Berlin, Germany, March 2015. https://hal-univ-rennes1.archives-ouvertes.fr/hal-01147733

Coutaz, J.: PAC: an object oriented model for implementing user interfaces. ACM SIGCHI Bull. 19(2), 37–41 (1987). https://doi.org/10.1145/36111.1045592. https://dl.acm.org/doi/10.1145/36111.1045592

10.

Coutrix, C., Nigay, L.: Mixed reality: a model of mixed interaction. In: Proceedings of the Working Conference on Advanced Visual Interfaces - AVI 2006, p. 43. ACM Press, Venezia (2006). https://doi.org/10.1145/1133265.1133274. http://portal.acm.org/citation.cfm?doid=1133265.1133274

11.

Dubois, E., Bortolaso, C., Appert, D., Gauffre, G.: An MDE-based framework to support the development of Mixed Interactive Systems. Sci. Comput. Program. 89, 199–221 (2014). https://doi.org/10.1016/j.scico.2013.03.007. https://linkinghub.elsevier.com/retrieve/pii/S0167642313000671

12.

Dubois, E., Gray, P., Nigay, L.: The Engineering of Mixed Reality Systems. Human-Computer Interaction Series, Springer, Heidelberg (2010). https://doi.org/10.1007/978-1-84882-733-2CrossRef

13.

Fuchs, P., Moreau, G., Guitton, P.: Virtual Reality: Concepts and Technologies. CRC Press (2011)

14.

Gibson, J.J.: The theory of affordances. In: Shaw, R.E., Bransford, J. (ed.) Perceiving, Acting, and Knowing: Toward an Ecological Psychology, pp. 67–82. Lawrence Erlbaum Associates, Hillsdale (1977). https://hal.archives-ouvertes.fr/hal-00692033

15.

Glaessgen, E., Stargel, D.: The digital twin paradigm for future NASA and US Air Force vehicles. In: 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference 20th AIAA/ASME/AHS Adaptive Structures Conference 14th AIAA, p. 1818 (2012)

16.

Greer, C., Burns, M., Wollman, D., Griffor, E.: Cyber-physical systems and internet of things. Technical report NIST SP 1900–202, National Institute of Standards and Technology, Gaithersburg, MD, March 2019. https://doi.org/10.6028/NIST.SP.1900-202. https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1900-202.pdf

17.

Grieves, M.W.: Virtually intelligent product systems: digital and physical twins. In: Complex Systems Engineering: Theory and Practice, pp. 175–200 (2019)

18.

Hussey, A., Carrington, D.: Comparing the MVC and PAC architectures: a formal perspective. IEE Proc. - Softw. Eng. 144(4), 224–236 (1997). https://doi.org/10.1049/ip-sen:19971512CrossRef

19.

Jacob, R.J., et al.: Reality-based interaction: a framework for post-WIMP interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2008, pp. 201–210. ACM, New York (2008). https://doi.org/10.1145/1357054.1357089. http://doi.acm.org/10.1145/1357054.1357089

20.

Kelaidonis, D., et al.: Virtualization and cognitive management of real world objects in the internet of things. In: 2012 IEEE International Conference on Green Computing and Communications, pp. 187–194. IEEE, Besancon, November 2012. https://doi.org/10.1109/GreenCom.2012.37. http://ieeexplore.ieee.org/document/6468312/

21.

Kim, K., Bölling, L., Haesler, S., Bailenson, J., Bruder, G., Welch, G.F.: Does a digital assistant need a body? The influence of visual embodiment and social behavior on the perception of intelligent virtual agents in AR, Munich, pp. 105–114 (2018). https://doi.org/10.1109/ISMAR.2018.00039

22.

Lacoche, J., Le Chenechal, M., Villain, E., Foulonneau, A.: Model and tools for integrating iot into mixed reality environments: towards a virtual-real seamless continuum. In: ICAT-EGVE 2019 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments. Tokyo, Japan, September 2019. https://hal.archives-ouvertes.fr/hal-02332096

23.

Lin, S., Cheng, H.F., Li, W., Huang, Z., Hui, P., Peylo, C.: Ubii: physical world interaction through augmented reality. IEEE Trans. Mob. Comput. 16(3), 872–885 (2017). https://doi.org/10.1109/TMC.2016.2567378CrossRef

24.

Liu, W.: Natural user interface- next mainstream product user interface. In: 2010 IEEE 11th International Conference on Computer-Aided Industrial Design Conceptual Design 1, vol. 1, pp. 203–205, November 2010. https://doi.org/10.1109/CAIDCD.2010.5681374

25.

MacWilliams, A., Sandor, C., Wagner, M., Bauer, M., Klinker, G., Bruegge, B.: Herding sheep: live system development for distributed augmented reality. In: Proceedings of the 2Nd IEEE/ACM International Symposium on Mixed and Augmented Reality, ISMAR 2003, p. 123. IEEE Computer Society, Washington, DC (2003). http://dl.acm.org/citation.cfm?id=946248.946803

26.

Marchand, E., Uchiyama, H., Spindler, F.: Pose estimation for augmented reality: a hands-on survey. IEEE Trans. Vis. Comput. Graph. 22(12), 2633–2651 (2016). https://doi.org/10.1109/TVCG.2015.2513408. http://ieeexplore.ieee.org/document/7368948/

27.

Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. E77-D (1994)

28.

Nitti, M., Pilloni, V., Colistra, G., Atzori, L.: The virtual object as a major element of the internet of things: a survey. IEEE Commun. Surv. Tutor. 18(2), 1228–1240 (2016). https://doi.org/10.1109/COMST.2015.2498304. http://ieeexplore.ieee.org/document/7320954/

29.

Norman, D.A.: The Psychology of Everyday Things. (The Design of Everyday Things). Basic Books (1988)

30.

Norman, D.A.: Affordance, conventions, and design. Interactions 6(3), 38–43 (1999). https://doi.org/10.1145/301153.301168. http://doi.acm.org/10.1145/301153.301168

31.

Norman, D.A.: Natural user interfaces are not natural. Interactions 17(3), 6–10 (2010). https://doi.org/10.1145/1744161.1744163. http://doi.acm.org/10.1145/1744161.1744163

32.

Pfeiffer, T., Pfeiffer-Leßmann, N.: Virtual prototyping of mixed reality interfaces with internet of things (IoT) connectivity. i-com 17(2), 179–186 (2018). https://doi.org/10.1515/icom-2018-0025. http://www.degruyter.com/view/j/icom.2018.17.issue-2/icom-2018-0025/icom-2018-0025.xml

33.

Poslad, S.: Ubiquitous Computing: Smart Devices, Environments and Interactions. Wiley, Hoboken (2011)

34.

Sánchez López, T., Ranasinghe, D.C., Harrison, M., Mcfarlane, D.: Adding sense to the internet of things. Pers.Ubiquit. Comput. 16(3), 291–308 (2012). https://doi.org/10.1007/s00779-011-0399-8. http://dx.doi.org/10.1007/s00779-011-0399-8

35.

Turner, P.: Affordance as context. Interact. Comput. 17(6), 787–800 (2005). https://doi.org/10.1016/j.intcom.2005.04.003CrossRef

36.

Uimonen, M.: Accessing BIM-related information through AR, p. 2 (2018)

37.

Walther, S.: ASP.NET 3.5 unleashed. Sams, Indianapolis, Ind (2008). oCLC: ocn180989631

Title: Unified Model and Framework for Interactive Mixed Entity Systems
Authors: Guillaume Bataille
Valérie Gouranton
Jérémy Lacoche
Danielle Pelé
Bruno Arnaldi
Publisher: Springer International Publishing
Book: Computer Vision, Imaging and Computer Graphics Theory and Applications
Print ISBN: 978-3-030-94892-4

Electronic ISBN: 978-3-030-94893-1

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-030-94893-1_1

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner