Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Beyond Usability: Methodologies and Bias – Surveying the Surveys Read chapter Read first chapter

2019 | OriginalPaper | Chapter

AR Contents Superimposition on Walls and Persons

Authors : João M. F. Rodrigues, Ricardo J. M. Veiga, Roman Bajireanu, Roberto Lam, Pedro J. S. Cardoso, Paulo Bica

Published in: Universal Access in Human-Computer Interaction. Theory, Methods and Tools

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

When it comes to visitors’ experiences at museums and heritage attractions, objects speak for themselves. With the aim of enhancing a traditional museum visit, a mobile Augmented Reality (AR) framework was developed during the M5SAR project. This paper presents two modules, the wall and human shape segmentation with AR content superimposition. The first, wall segmentation, is achieved by using a BRISK descriptor and geometric information, having the wall delimited, and the AR contents superposed over the detected wall contours. The second module, person segmentation, is achieved by using an OpenPose model, which computes the body joints. These joints are then combined with volumes to achieve AR clothes content superimposition. This paper shows the usage of both methods in a real museum environment.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Editor of O & M Virtual Environments for the Training of People with Visual Impairment
next chapter Gaming Background Influence on VR Performance and Comfort: A Study Using Different Navigation Metaphors
Footnotes
1
Mobile Five Senses Augmented Reality System for Museums, financed by CRESC ALGARVE2020, PORTUGAL2020 and FEDER.
 
Literature
1.
Araki, N., Muraoka, Y.: Follow-the-trial-fitter: real-time dressing without undressing. In: Proceedings of IEEE Conference on Digital Information Management, London, UK, pp. 33–38 (2008)
2.
3.
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Comput. Graph. Appl. 21(6), 34–47 (2001)CrossRef
4.
5.
Bajireanu, R., et al.: Mobile human shape superimposition: an initial approach using OpenPose. In: Proceedings 18th International Conference on Applied Computer Science (2018)
6.
Bartoli, A., Sturm, P.: Structure-from-motion using lines: representation, triangulation, and bundle adjustment. Comput. Vis. Image Underst. 100(3), 416–441 (2005)CrossRef
7.
Bouguet, J.-Y.: Pyramidal implementation of the affine Lucas Kanade feature tracker description of the algorithm. Intel Corporation 5(1–10), 4 (2001)
8.
Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)CrossRef
9.
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y.: Openpose: realtime multi-person 2D pose estimation using part affinity fields. arXiv preprint arXiv:​1812.​08008 (2018)
10.
Cao, Z., Simon, T., Wei, S.-E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: CVPR, vol. 1, no. 2, p. 7 (2017)
11.
12.
Cheng, K.-H., Tsai, C.-C.: Affordances of augmented reality in science learning: suggestions for future research. J. Sci. Educ. Technol. 22(4), 449–462 (2013)CrossRef
13.
Durrant-Whyte, H., Bailey, T.: Simultaneous localization and mapping: part I. IEEE Rob. Autom. Mag. 13(2), 99–110 (2006)CrossRef
14.
Elqursh, A., Elgammal, A.: Line-based relative pose estimation. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp. 3049–3056. IEEE (2011)
15.
Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2018)CrossRef
16.
17.
Engel, J., Sturm, J., Cremers, D.: Semi-dense visual odometry for a monocular camera. In: Proceedings IEEE International Conference on Computer Vision, pp. 1449–1456 (2013)
18.
Erra, U., Scanniello, G., Colonnese, V.: Exploring the effectiveness of an augmented reality dressing room. Multimedia Tools Appl., 1–31 (2018)
19.
20.
Fang, H., Xie, S., Tai, Y.-W., Lu, C.: RMPE: regional multi-person pose estimation. In: Proceedings IEEE International Conference on Computer Vision, vol. 2 (2017)
21.
Gimeno, J., Portales, C., Coma, I., Fernandez, M., Martinez, B.: Combining traditional and indirect augmented reality for indoor crowded environments. A case study on the casa batlló museum. Comput. Graph. 69, 92–103 (2017)CrossRef
22.
Girshick, R.: Fast R-CNN. In: Proceedings IEEE Conference on Computer Vision, pp. 1440–1448 (2015)
23.
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)
24.
25.
Gupta, S., Arbeláez, P., Girshick, R., Malik, J.: Indoor scene understanding with RGB-D images: bottom-up segmentation, object detection and semantic segmentation. Int. J. Comput. Vis. 112(2), 133–149 (2015)MathSciNetCrossRef
26.
Haines, O., Calway, A.: Detecting planes and estimating their orientation from a single image. In: BMVC, pp. 1–11 (2012)
27.
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
28.
Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:​1704.​04861 (2017)
29.
Huang, J., et al.: Speed/Accuracy trade-offs for modern convolutional object detectors. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, vol. 4, Honolulu, HI, USA, pp. 3296–3297 (2017)
30.
Hulik, R., Spanel, M., Smrz, P., Materna, Z.: Continuous plane detection in point-cloud data based on 3D Hough transform. J. Vis. Commun. Image Represent. 25(1), 86–97 (2014)CrossRef
31.
32.
Isıkdogan, F., Kara, G.: A real time virtual dressing room application using Kinect. Computer Vision Course Project (2012)
33.
Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)
34.
35.
36.
37.
38.
Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: binary robust invariant scalable keypoints. In: Proceedings IEEE International Conference on Computer Vision, pp. 2548–2555. IEEE (2011)
39.
Mallya, A., Lazebnik, S.: Learning informative edge maps for indoor scene layout prediction. In: Proceedings IEEE International Conference on Computer Vision, pp. 936–944 (2015)
40.
41.
Muja, M., Lowe, D.G.: Fast matching of binary features. In: Proceedings 9th Conference Computer and Robot Vision, pp. 404–410. IEEE (2012)
42.
Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Rob. 31(5), 1147–1163 (2015)CrossRef
43.
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)CrossRef
44.
Papandreou, G., Zhu, T., Chen, L.-C., Gidaris, S., Tompson, J., Murphy, K.: PersonLab: person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model. arXiv preprint arXiv:​1803.​08225 (2018)
45.
46.
Portales, C., Vinals, M.J., Alonso-Monasterio, P.: AR-immersive cinema at the aula natura visitors center. IEEE MultiMedia 17(4), 8–15 (2010)CrossRef
47.
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
48.
Ren, Z., Sudderth, E.B.: Three-dimensional object detection and layout prediction using clouds of oriented gradients. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp. 1525–1533 (2016)
49.
Rodrigues, J.M.F., et al.: Adaptive card design UI implementation for an augmented reality museum application. In: Proceedings 11th International Conference on Universal Access in Human-Computer Interaction (2017)
50.
51.
Serrão, M., et al.: Computer vision and GIS for the navigation of blind persons in buildings. Univ. Access Inf. Soc. 14(1), 67–80 (2015)CrossRef
52.
Shi, J., Tomasi, C.: Good features to track. Technical report, Cornell University (1993)
53.
Tareen, S.A.K., Saleem, Z.: A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. In: Proceedings International Conference on Computing, Mathematics and Engineering Technologies, pp. 1–10. IEEE (2018)
54.
Tateno, K., Tombari, F., Laina, I., Navab, N.: CNN-SLAM: real-time dense monocular SLAM with learned depth prediction. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, vol. 2 (2017)
55.
Tome, D., Russell, C., Agapito, L.: Lifting from the deep: convolutional 3D pose estimation from a single image. In: Proceedings IEEE Conference Computer Vision and Pattern Recognition, pp. 2500–2509 (2017)
56.
57.
58.
Vainstein, N., Kuflik, T., Lanir, J.: Towards using mobile, head-worn displays in cultural heritage: user requirements and a research agenda. In: Proceedings 21st International Conference on Intelligent User Interfaces, pp. 327–331. ACM (2016)
59.
Veiga, R.J.M., Bajireanu, R., Pereira, J.A.R., Sardo, J.D.P., Cardoso, P.J.S., Rodrigues, J.M.F.: Indoor environment and human shape detection for augmented reality: an initial study. In: Proceedings 23rd Portuguese Conference Pattern Recognition, p. 21 (2017)
60.
Veiga, R.J.M., Pereira, J.A.R., Sardo, J.D.P., Bajireanu, R., Cardoso, P.J.S., Rodrigues, J.M.F.: Augmented reality indoor environment detection: proof-of-concept. In: Proceedings Applied Mathematics And Computer Science (2018)
Metadata
Title
AR Contents Superimposition on Walls and Persons
Authors
João M. F. Rodrigues
Ricardo J. M. Veiga
Roman Bajireanu
Roberto Lam
Pedro J. S. Cardoso
Paulo Bica
Copyright Year
2019
Book
Universal Access in Human-Computer Interaction. Theory, Methods and Tools

Print ISBN: 978-3-030-23559-8

Electronic ISBN: 978-3-030-23560-4

Copyright Year: 2019

DOI
https://doi.org/10.1007/978-3-030-23560-4_46