Skip to main content

2021 | OriginalPaper | Buchkapitel

Towards Fully-Synthetic Training for Industrial Applications

verfasst von : Christopher Mayershofer, Tao Ge, Johannes Fottner

Erschienen in: LISS 2020

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper proposes a scalable approach for synthetic image generation of industrial objects leveraging Blender for image rendering. In addition to common components in synthetic image generation research, three novel features are presented: First, we model relations between target objects and randomly apply those during scene generation (Object Relation Modelling (ORM)). Second, we extend the idea of distractors and create Object-alike Distractors (OAD), resembling the textural appearance (i.e. material and size) of target objects. And third, we propose a Mixed-lighting Illumination (MLI), combining global and local light sources to automatically create a diverse illumination of the scene. In addition to the image generation approach we create an industry-centered dataset for evaluation purposes. Experiments show, that our approach enables fully synthetic training of object detectors for industrial use-cases. Moreover, an ablation study provides evidence on the performance boost in object detection when using our novel features.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat S. Ren, K. He, R.B. Girshik, J. Sun, Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015) S. Ren, K. He, R.B. Girshik, J. Sun, Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015)
2.
Zurück zum Zitat J. Redmon, A. Farhadi, Yolo9000: Better, Faster, Stronger (2016) J. Redmon, A. Farhadi, Yolo9000: Better, Faster, Stronger (2016)
3.
Zurück zum Zitat T. Lin, P. Goyal, R.B. Girshick, K. He, P. Dollár, Focal Loss for Dense Object Detection (2017) T. Lin, P. Goyal, R.B. Girshick, K. He, P. Dollár, Focal Loss for Dense Object Detection (2017)
4.
Zurück zum Zitat M. Tan, R. Pang, Q.V. Le, EfficientDet: Scalable and Efficient Object Detection (2019) M. Tan, R. Pang, Q.V. Le, EfficientDet: Scalable and Efficient Object Detection (2019)
5.
Zurück zum Zitat K. He, G. Gkioxari, P. Dollár, R.B. Girshick, Mask R-CNN (2017) K. He, G. Gkioxari, P. Dollár, R.B. Girshick, Mask R-CNN (2017)
6.
Zurück zum Zitat W. Chen, X. Gong, X. Liu, Q. Zhang, Y. Li, Z. Wang, FasterSeg: Searching for Faster Real-Time Semantic Segmentation (2020) W. Chen, X. Gong, X. Liu, Q. Zhang, Y. Li, Z. Wang, FasterSeg: Searching for Faster Real-Time Semantic Segmentation (2020)
7.
Zurück zum Zitat Z. Su, M. Ye, G. Zhang, L. Dai, J. Sheng, Cascade Feature Aggregation for Human Pose Estimation (2019) Z. Su, M. Ye, G. Zhang, L. Dai, J. Sheng, Cascade Feature Aggregation for Human Pose Estimation (2019)
8.
Zurück zum Zitat A. Bulat, J. Kossaifi, G. Tzimiropoulos, M. Pantic, Toward Fast and Accurate Human Pose Estimation Via Soft-Gated Skip Connections (2020) A. Bulat, J. Kossaifi, G. Tzimiropoulos, M. Pantic, Toward Fast and Accurate Human Pose Estimation Via Soft-Gated Skip Connections (2020)
9.
Zurück zum Zitat C. Godard, O. Mac Aodha, G.J. Brostow, Unsupervised Monocular Depth Estimation with Left-Right Consistency (2016) C. Godard, O. Mac Aodha, G.J. Brostow, Unsupervised Monocular Depth Estimation with Left-Right Consistency (2016)
10.
Zurück zum Zitat J.H. Lee, M. Han, D.W. Ko, I.H. Suh, From Big to Small: Multi-scale Local Planar Guidance for Monocular Depth Estimation (2019) J.H. Lee, M. Han, D.W. Ko, I.H. Suh, From Big to Small: Multi-scale Local Planar Guidance for Monocular Depth Estimation (2019)
11.
Zurück zum Zitat A. Geiger, P. Lenz, C. Stiller, R. Urtasun, Vision Meets Robotics: The KITTI Dataset (2013) A. Geiger, P. Lenz, C. Stiller, R. Urtasun, Vision Meets Robotics: The KITTI Dataset (2013)
12.
Zurück zum Zitat P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine, V. Vasudevan, W. Han, J. Ngiam, H. Zhao, A. Timofeev, S. Ettinger, M. Krivokon, A. Gao, A. Joshi, Y. Zhang, J. Shlens, Z. Chen, D. Anguelov, Scalability in Perception for Autonomous Driving: Waymo Open Dataset (2019) P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine, V. Vasudevan, W. Han, J. Ngiam, H. Zhao, A. Timofeev, S. Ettinger, M. Krivokon, A. Gao, A. Joshi, Y. Zhang, J. Shlens, Z. Chen, D. Anguelov, Scalability in Perception for Autonomous Driving: Waymo Open Dataset (2019)
13.
Zurück zum Zitat M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, A. Zisserman, The PASCAL Visual Object Classes (VOC) challenge (2010) M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, A. Zisserman, The PASCAL Visual Object Classes (VOC) challenge (2010)
14.
Zurück zum Zitat T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C.L. Zitnick, P. Dollár, Microsoft coco: Common Objects in Context (2014) T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C.L. Zitnick, P. Dollár, Microsoft coco: Common Objects in Context (2014)
15.
Zurück zum Zitat H. Noh, A. Araujo, J. Sim, B. Han, Image Retrieval with Deep Local Features and Attention-Based Keypoints (2016) H. Noh, A. Araujo, J. Sim, B. Han, Image Retrieval with Deep Local Features and Attention-Based Keypoints (2016)
16.
Zurück zum Zitat F.E. Nowruzi, P. Kapoor, D. Kolhatkar, F. Al Hassanat, R. Laganiere, J. Rebut, How Much Real Data do we Actually Need: Analyzing Object Detection Performance Using Synthetic and Real Data (2019) F.E. Nowruzi, P. Kapoor, D. Kolhatkar, F. Al Hassanat, R. Laganiere, J. Rebut, How Much Real Data do we Actually Need: Analyzing Object Detection Performance Using Synthetic and Real Data (2019)
17.
Zurück zum Zitat X. Peng, K. Saenko, Synthetic to Real Adaptation with Deep Generative Correlation Alignment Networks (2017) X. Peng, K. Saenko, Synthetic to Real Adaptation with Deep Generative Correlation Alignment Networks (2017)
18.
Zurück zum Zitat D. Dwibedi, I. Misra, M. Hebert, Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection (2017) D. Dwibedi, I. Misra, M. Hebert, Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection (2017)
19.
Zurück zum Zitat M. Johnson-Roberson, C. Barto, R. Mehta, S.N. Sridhar, K. Rosaen, R. Vasudevan, Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks? (2017) M. Johnson-Roberson, C. Barto, R. Mehta, S.N. Sridhar, K. Rosaen, R. Vasudevan, Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks? (2017)
20.
Zurück zum Zitat A. Prakash, S. Boochoon, M. Brophy, D. Acuna, E. Cameracci, G. State, O. Shapira, S. Birchfield, Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data (2018) A. Prakash, S. Boochoon, M. Brophy, D. Acuna, E. Cameracci, G. State, O. Shapira, S. Birchfield, Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data (2018)
21.
Zurück zum Zitat S. Hinterstoisser, O. Pauly, H. Heibel, M. Marek, M. Bokeloh, An Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance an Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance Detection (2019) S. Hinterstoisser, O. Pauly, H. Heibel, M. Marek, M. Bokeloh, An Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance an Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance Detection (2019)
22.
Zurück zum Zitat X.B. Peng, M. Andrychowicz, W. Zaremba, P. Abbeel, Sim-to-Real Transfer of Robotic Control with Dynamics Randomization (2018) X.B. Peng, M. Andrychowicz, W. Zaremba, P. Abbeel, Sim-to-Real Transfer of Robotic Control with Dynamics Randomization (2018)
23.
Zurück zum Zitat M. Rad, M. Oberweger, V. Lepetit, Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Images (2018) M. Rad, M. Oberweger, V. Lepetit, Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Images (2018)
24.
Zurück zum Zitat J. Borrego, A. Dehban, R. Figueiredo, P. Moreno, A. Bernardino, J. Santos-Victor, Applying Domain Randomization to Synthetic Data for Object Category Detection (2018) J. Borrego, A. Dehban, R. Figueiredo, P. Moreno, A. Bernardino, J. Santos-Victor, Applying Domain Randomization to Synthetic Data for Object Category Detection (2018)
25.
Zurück zum Zitat X. Pan, P. Luo, J. Shi, X. Tang, Two at Once: Enhancing Learning and Generalization Capacities Via ibn-net (2018) X. Pan, P. Luo, J. Shi, X. Tang, Two at Once: Enhancing Learning and Generalization Capacities Via ibn-net (2018)
26.
Zurück zum Zitat G. Yang, H. Xia, M. Ding, Z. Ding, Bi-directional Generation for Unsupervised Domain Adaptation (2020) G. Yang, H. Xia, M. Ding, Z. Ding, Bi-directional Generation for Unsupervised Domain Adaptation (2020)
27.
Zurück zum Zitat S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, K. Bousmalis, Sim-to-Real via Sim-to-Sim Data-Efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks (2019) S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, K. Bousmalis, Sim-to-Real via Sim-to-Sim Data-Efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks (2019)
28.
Zurück zum Zitat S. Thalhammer, K. Park, T. Patten, M. Vincze, W. Kropatsch, Sydd Synthetic Depth Data Randomization for Object Detection Using Domain-Relevant Background (2019) S. Thalhammer, K. Park, T. Patten, M. Vincze, W. Kropatsch, Sydd Synthetic Depth Data Randomization for Object Detection Using Domain-Relevant Background (2019)
29.
Zurück zum Zitat J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, P. Abbeel, Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World (2017) J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, P. Abbeel, Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World (2017)
30.
Zurück zum Zitat N. Mayer, E. Ilg, P. Fischer, C. Hazirbas, D. Cremers, A. Dosovitskiy, T. Brox, What makes good synthetic training data for learning disparity and optical flow estimation? Int. J. Comput. Vis. 126(9), 942–960 (2018)CrossRef N. Mayer, E. Ilg, P. Fischer, C. Hazirbas, D. Cremers, A. Dosovitskiy, T. Brox, What makes good synthetic training data for learning disparity and optical flow estimation? Int. J. Comput. Vis. 126(9), 942–960 (2018)CrossRef
31.
Zurück zum Zitat J. Tremblay, A. Prakash, D. Acuna, M. Brophy, V. Jampani, C. Anil, T. To, E. Cameracci, S. Boochoon, S. Birchfield, Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization (2018) J. Tremblay, A. Prakash, D. Acuna, M. Brophy, V. Jampani, C. Anil, T. To, E. Cameracci, S. Boochoon, S. Birchfield, Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization (2018)
32.
Zurück zum Zitat B. Mehta, M. Diaz, G. Florian, C. J. Pal, L. Paull, Active Domain Randomization (2019) B. Mehta, M. Diaz, G. Florian, C. J. Pal, L. Paull, Active Domain Randomization (2019)
33.
Zurück zum Zitat I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, W. Zaremba, L. Zhang, Solving Rubik’s Cube with a Robot Hand (2019) I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, W. Zaremba, L. Zhang, Solving Rubik’s Cube with a Robot Hand (2019)
34.
Zurück zum Zitat Y. Movshovitz-Attias, T. Kanade, Y. Sheikh, How Useful is Photo-Realistic Rendering for Visual Learning? (2016) Y. Movshovitz-Attias, T. Kanade, Y. Sheikh, How Useful is Photo-Realistic Rendering for Visual Learning? (2016)
35.
Zurück zum Zitat J. Redmon, A. Farhadi, Yolov3: An Incremental Improvement (2018) J. Redmon, A. Farhadi, Yolov3: An Incremental Improvement (2018)
Metadaten
Titel
Towards Fully-Synthetic Training for Industrial Applications
verfasst von
Christopher Mayershofer
Tao Ge
Johannes Fottner
Copyright-Jahr
2021
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-33-4359-7_53

Premium Partner