Skip to main content

2016 | OriginalPaper | Buchkapitel

Learning and Detecting Objects with a Mobile Robot to Assist Older Adults in Their Homes

verfasst von : Markus Vincze, Markus Bajones, Markus Suchi, Daniel Wolf, Astrid Weiss, David Fischinger, Paloma da la Puente

Erschienen in: Computer Vision – ECCV 2016 Workshops

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Older adults reported that a robot in their homes would be of great help if it could find objects that users regularly search for. We propose an interactive method to learn objects directly with the user and the robot and then use the RGB-D model to search for the object in the scene. The robot presents a turntable to the user for rotating the part in front of its camera and obtain a full 3D model. The user is asked to turn the object upside down and the two half-models are merged. The model is then used at predefined search locations for detecting the object on tables or other horizontal surfaces. Experiments in three environments, up to 14 objects and a total of 1080 scenes indicate that present detection methods need to be considerably improved to provide a good service to users. We analyse the results and contribute to the discussion on how to overcome limited image quality and resolution by exploiting the robotic system.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Kragic, D., Vincze, M.: Vision for robotics. Foundations Trends Robot. 1(1), 1–78 (2010)CrossRef Kragic, D., Vincze, M.: Vision for robotics. Foundations Trends Robot. 1(1), 1–78 (2010)CrossRef
2.
Zurück zum Zitat Fazekas, G., Tth, A., Rumeau, P., Zsiga, K., Pilissy, T., Dupourque, V.: Cognitive-care robot for elderly assistance: preliminary results of tests with users in their homes. In: AAL Forum, Netherlands (2012) Fazekas, G., Tth, A., Rumeau, P., Zsiga, K., Pilissy, T., Dupourque, V.: Cognitive-care robot for elderly assistance: preliminary results of tests with users in their homes. In: AAL Forum, Netherlands (2012)
3.
Zurück zum Zitat Panek, P., Beck, C., Edelmayer, G., Mayer, P., Rauhala, M., Zagler, W.L.: Connecting AAL devices and systems to improve service delivery. In: AAL Forum: Broader, p. 2014. Romania, Bigger, Better - AAL solutions for Europe, Palace of Parliament, Bucharest (2014) Panek, P., Beck, C., Edelmayer, G., Mayer, P., Rauhala, M., Zagler, W.L.: Connecting AAL devices and systems to improve service delivery. In: AAL Forum: Broader, p. 2014. Romania, Bigger, Better - AAL solutions for Europe, Palace of Parliament, Bucharest (2014)
4.
Zurück zum Zitat Bedaf, S., Gelderblom, G.J., de Witte, L., Syrdal, D., Lehnmann, H., Amirabdollahian, F., Dautenhahn, K.: Selecting services for a service robot - evaluating the problematic activities threatening the independence of elderly persons. In: ICORR (2013) Bedaf, S., Gelderblom, G.J., de Witte, L., Syrdal, D., Lehnmann, H., Amirabdollahian, F., Dautenhahn, K.: Selecting services for a service robot - evaluating the problematic activities threatening the independence of elderly persons. In: ICORR (2013)
5.
Zurück zum Zitat Ullberg, J., Loutfi, A., Pecora, F.: A customizable approach for monitoring activities of elderly users in their homes. In: Mazzeo, P.L., Spagnolo, P., Moeslund, T.B. (eds.) AMMDS 2014. LNCS, vol. 8703, pp. 13–25. Springer, Heidelberg (2014) Ullberg, J., Loutfi, A., Pecora, F.: A customizable approach for monitoring activities of elderly users in their homes. In: Mazzeo, P.L., Spagnolo, P., Moeslund, T.B. (eds.) AMMDS 2014. LNCS, vol. 8703, pp. 13–25. Springer, Heidelberg (2014)
6.
Zurück zum Zitat Glende, S., Conrad, I., Krezdorn, L., Klemcke, S., Krtzel, C.: Increasing acceptance of assistive robotics for older people through marketing strategies based on stakeholder needs. Int. J. Soc. Robot. 8, 355–369 (2016)CrossRef Glende, S., Conrad, I., Krezdorn, L., Klemcke, S., Krtzel, C.: Increasing acceptance of assistive robotics for older people through marketing strategies based on stakeholder needs. Int. J. Soc. Robot. 8, 355–369 (2016)CrossRef
7.
Zurück zum Zitat Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef
8.
Zurück zum Zitat Endres, F., Hess, J., Sturm, J., Cremers, D., Burgard, W.: 3-D mapping with an RGB-D camera. IEEE Trans. Robot. 30(1), 177–187 (2014)CrossRef Endres, F., Hess, J., Sturm, J., Cremers, D., Burgard, W.: 3-D mapping with an RGB-D camera. IEEE Trans. Robot. 30(1), 177–187 (2014)CrossRef
9.
Zurück zum Zitat Collet, A., Martinez, M., Srinivasa, S.S.: The moped framework: object recognition and pose estimation for manipulation. Int. J. Rob. Res. 30(10), 1284–1306 (2011)CrossRef Collet, A., Martinez, M., Srinivasa, S.S.: The moped framework: object recognition and pose estimation for manipulation. Int. J. Rob. Res. 30(10), 1284–1306 (2011)CrossRef
10.
Zurück zum Zitat Huber, D.F., Hebert, M.: Fully automatic registration of multiple 3D data sets. Image Vis. Comput. 21(7), 637–650 (2003)CrossRef Huber, D.F., Hebert, M.: Fully automatic registration of multiple 3D data sets. Image Vis. Comput. 21(7), 637–650 (2003)CrossRef
11.
Zurück zum Zitat Fantoni, S., Castellani, U., Fusiello, A.: Accurate and automatic alignment of range surfaces. In: Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 73–80 (2012) Fantoni, S., Castellani, U., Fusiello, A.: Accurate and automatic alignment of range surfaces. In: Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 73–80 (2012)
12.
Zurück zum Zitat Weise, T., Wismer, T., Leibe, B., Gool, L.V.: Online loop closure for real-time interactive 3D scanning. Comput. Vis. Image Underst. 115(5), 635–648 (2011)CrossRef Weise, T., Wismer, T., Leibe, B., Gool, L.V.: Online loop closure for real-time interactive 3D scanning. Comput. Vis. Image Underst. 115(5), 635–648 (2011)CrossRef
13.
Zurück zum Zitat Dimashova, M., Lysenkov, I., Rabaud, V., Eruhimov, V.: Tabletop object scanning with an RGB-D sensor. In: SPME (2013) Dimashova, M., Lysenkov, I., Rabaud, V., Eruhimov, V.: Tabletop object scanning with an RGB-D sensor. In: SPME (2013)
14.
Zurück zum Zitat Krainin, M., Curless, B., Fox, D.: Autonomous generation of complete 3D object models using next best view manipulation planning. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 5031–5037, May 2011 Krainin, M., Curless, B., Fox, D.: Autonomous generation of complete 3D object models using next best view manipulation planning. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 5031–5037, May 2011
15.
Zurück zum Zitat Weise, T., Wismer, T., Leibe, B., Gool, L.V.: In-hand scanning with online loop closure. In: IEEE ICCV Workshop (2009) Weise, T., Wismer, T., Leibe, B., Gool, L.V.: In-hand scanning with online loop closure. In: IEEE ICCV Workshop (2009)
16.
Zurück zum Zitat Stueckler, J., Behnke, S.: Multi-resolution surfel maps for efficient dense 3D modeling and tracking. J. Vis. Commun. Image Represent. 25(1), 137–147 (2014)CrossRef Stueckler, J., Behnke, S.: Multi-resolution surfel maps for efficient dense 3D modeling and tracking. J. Vis. Commun. Image Represent. 25(1), 137–147 (2014)CrossRef
17.
Zurück zum Zitat Aldoma, A., Marton, Z.C., Tombari, F., Wohlkinger, W., Potthast, C., Zeisl, B., Rusu, R.B., Gedikli, S.: Using the point cloud library for 3D object recognition and 6dof pose estimation. IEEE Robot. Autom. Mag. 9, 80–91 (2012)CrossRef Aldoma, A., Marton, Z.C., Tombari, F., Wohlkinger, W., Potthast, C., Zeisl, B., Rusu, R.B., Gedikli, S.: Using the point cloud library for 3D object recognition and 6dof pose estimation. IEEE Robot. Autom. Mag. 9, 80–91 (2012)CrossRef
18.
Zurück zum Zitat Lysenkov, I., Eruhimov, V., Bradski, G.: Recognition and pose estimation of rigid transparent objects with a kinect sensor. In: Proceedings of Robotics: Science and Systems, Sydney, Australia, July 2012 Lysenkov, I., Eruhimov, V., Bradski, G.: Recognition and pose estimation of rigid transparent objects with a kinect sensor. In: Proceedings of Robotics: Science and Systems, Sydney, Australia, July 2012
19.
Zurück zum Zitat Prankl, J., Aldoma, A., Svejda, A., Vincze, M.: RGB-D object modelling for object recognition and tracking. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 96–103, September 2015 Prankl, J., Aldoma, A., Svejda, A., Vincze, M.: RGB-D object modelling for object recognition and tracking. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 96–103, September 2015
20.
Zurück zum Zitat Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2011, pp. 127–136. IEEE Computer Society (2011) Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2011, pp. 127–136. IEEE Computer Society (2011)
21.
Zurück zum Zitat Aldoma, A., Tombari, F., Stefano, L.D., Vincze, M.: A global hypothesis verification framework for 3D object recognition in clutter. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2015) Aldoma, A., Tombari, F., Stefano, L.D., Vincze, M.: A global hypothesis verification framework for 3D object recognition in clutter. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2015)
22.
Zurück zum Zitat Aldoma, A., Tombari, F., Di Stefano, L., Vincze, M.: A global hypotheses verification method for 3D object recognition. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 511–524. Springer, Heidelberg (2012)CrossRef Aldoma, A., Tombari, F., Di Stefano, L., Vincze, M.: A global hypotheses verification method for 3D object recognition. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 511–524. Springer, Heidelberg (2012)CrossRef
23.
Zurück zum Zitat Aldoma, A., Tombari, F., Rusu, R.B., Vincze, M.: OUR-CVFH – oriented, unique and repeatable clustered viewpoint feature histogram for object recognition and 6DOF pose estimation. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM and OAGM 2012. LNCS, vol. 7476, pp. 113–122. Springer, Heidelberg (2012)CrossRef Aldoma, A., Tombari, F., Rusu, R.B., Vincze, M.: OUR-CVFH – oriented, unique and repeatable clustered viewpoint feature histogram for object recognition and 6DOF pose estimation. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM and OAGM 2012. LNCS, vol. 7476, pp. 113–122. Springer, Heidelberg (2012)CrossRef
24.
Zurück zum Zitat Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, pp. 1–4, 9–13 May 2011 Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, pp. 1–4, 9–13 May 2011
25.
Zurück zum Zitat Tombari, F., Salti, S., Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 356–369. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15558-1_26 CrossRef Tombari, F., Salti, S., Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 356–369. Springer, Heidelberg (2010). doi:10.​1007/​978-3-642-15558-1_​26 CrossRef
26.
Zurück zum Zitat Andreopoulos, A., Hasler, S., Wersing, H., Janssen, H., Tsotsos, J., Korner, E.: Active 3D object localization using a humanoid robot. IEEE Trans. Rob. 27(1), 47–64 (2011)CrossRef Andreopoulos, A., Hasler, S., Wersing, H., Janssen, H., Tsotsos, J., Korner, E.: Active 3D object localization using a humanoid robot. IEEE Trans. Rob. 27(1), 47–64 (2011)CrossRef
27.
Zurück zum Zitat Bajones, M., Wolf, D., Prankl, J., Vincze, M.: Where to look first? Behaviour control for fetch-and-carry missions of service robots. In: Austrian Robotics Workshop(2014) Bajones, M., Wolf, D., Prankl, J., Vincze, M.: Where to look first? Behaviour control for fetch-and-carry missions of service robots. In: Austrian Robotics Workshop(2014)
28.
Zurück zum Zitat Wolf, D., Prankl, J., Vincze, M.: Enhancing semantic segmentation for robotics: the power of 3-D entangled forests. IEEE Robot. Autom. Lett. 1(1), 49–56 (2016)CrossRef Wolf, D., Prankl, J., Vincze, M.: Enhancing semantic segmentation for robotics: the power of 3-D entangled forests. IEEE Robot. Autom. Lett. 1(1), 49–56 (2016)CrossRef
29.
Zurück zum Zitat Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012)CrossRef Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012)CrossRef
30.
Zurück zum Zitat Fäulhammer, T., Ambrus, R., Burbridge, C., Zillich, M., Folkesson, J., Hawes, N., Jensfelt, P., Vincze, M.: Autonomous learning of object models on a mobile robot. IEEE Robot. Autom. Lett. 2(1), 26–33 (2016)CrossRef Fäulhammer, T., Ambrus, R., Burbridge, C., Zillich, M., Folkesson, J., Hawes, N., Jensfelt, P., Vincze, M.: Autonomous learning of object models on a mobile robot. IEEE Robot. Autom. Lett. 2(1), 26–33 (2016)CrossRef
31.
Zurück zum Zitat Makihara, Y., Takizawa, M., Shirai, Y., Miura, J., Shimada, N.: Object recognition supported by user interaction for service robots. In: 16th International Conference on Pattern Recognition, Proceedings, vol. 3, pp. 561–564 (2002) Makihara, Y., Takizawa, M., Shirai, Y., Miura, J., Shimada, N.: Object recognition supported by user interaction for service robots. In: 16th International Conference on Pattern Recognition, Proceedings, vol. 3, pp. 561–564 (2002)
Metadaten
Titel
Learning and Detecting Objects with a Mobile Robot to Assist Older Adults in Their Homes
verfasst von
Markus Vincze
Markus Bajones
Markus Suchi
Daniel Wolf
Astrid Weiss
David Fischinger
Paloma da la Puente
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-48881-3_22

Premium Partner