Top

Published in:

2019 | OriginalPaper | Chapter

Image Classification for Robotic Plastering with Convolutional Neural Network

Authors : Joshua Bard, Ardavan Bidgoli, Wei Wei Chi

Published in: Robotic Fabrication in Architecture, Art and Design 2018

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Inspecting robotically fabricated objects to detect and classify discrepancies between virtual target models and as-built realities is one of the challenges that faces robotic fabrication. Industrial-grade computer vision methods have been widely used to detect manufacturing flaws in mass production lines. However, in mass-customization, a versatile and robust method should be flexible enough to ignore construction tolerances while detecting specified flaws in varied parts. This study aims to leverage recent developments in machine learning and convolutional neural networks to improve the resiliency and accuracy of surface inspections in architectural robotics. Under a supervised learning scenario, the authors compared two approaches: (1) transfer learning on a general purpose Convolutional Neural Network (CNN) image classifier, and (2) design and train a CNN from scratch to detect and categorize flaws in a robotic plastering workflow. Both CNNs were combined with conventional search methods to improve the accuracy and efficiency of the system. A web-based graphical user interface and a real-time video projection method were also developed to facilitate user interactions and control over the workflow.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

next chapter Designing Natural Wood Log Structures with Stochastic Assembly and Deep Learning

Code for this project are available on GitHub (https://github.com/Ardibid/RoboticPlasteringCNN).

In the same year, at CVPR2014, Oquab et al. and Sharif et al. also addressed transfer learning and representation. For further information please look at (Oquab et al. 2014; Sharif Razavian et al. 2014).

AlexNet architecture might be confusing at the first sight since it has two parallel pipelines. However, the reason behind this dual pipeline is to train the model on two separate GPU simultaneously.

Markups are simple user-defined drawings, i.e. circles and crosses, that can be used to communicate with the system.

Although the authors first implemented Quad Tree search algorithm to compensate for the possible slow classification pipeline, the final model performance was good enough to provide near real-time experience. Accordingly, we opted for a grid search algorithm and avoided potential challenges that a Quad Tree search would introduce. The biggest drawback being its tendency to ignore small features in the initial steps of the search process when surveying large areas of the given image.

Amtsberg, F., Raspall, F., Trummer, A.: Digital-material feedback in architectural design. In: Ikeda, Y., Kaijima, S., Herr, C., Schnabel, M.A. (eds.) Proceedings of the 20th International Conference of the Association for Computer-Aided Architectural Design Research in Asia: Emerging Experience in Past, Present and Future of Digital Architecture, CAADRIA 2015, pp. 631–640. Daegu, South Korea (2015)

Bard, J., Blackwood, D., Sekhar, S., Brian, N.: Reality is interface: two motion capture case studies of human–machine collaboration in high-skill domain. Int. J. Architectural Comput. 14(4), 398–408 (2016a)CrossRef

Bard, J., Tursky, R., Jeffers, M.: RECONstruction. In: Reinhardt, D., Saunders, R., Burry, J. (eds.) Robotic Fabrication Architecture, Art and Design 2016, pp. 262–273. Springer, Switzerland (2016b)CrossRef

Bidgoli, A., Cardoso Llach, D.: Towards a motion grammar for robotic stereotomy. In: Ikeda, Y., Kaijima, S., Herr, C., Schnabel, M.A. (eds.) Proceedings of the 20th International Conference of the Association for Computer-Aided Architectural Design Research in Asia: Emerging Experience in Past, Present and Future of Digital Architecture, CAADRIA 2015, pp. 723–732. Daegu, South Korea (2015)

Dawson-Haggerty, M. (n.d.): Open ABB Driver. https://github.com/robotics/open_abb. Accessed 5 Oct 2017

Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: DeCAF: a deep convolutional activation feature for generic visual recognition. In: Proceedings of the 31st International Conference on Machine Learning, PMLR, vol. 32(1), pp. 647–655. ACM, New York (2014)

Egmont-Petersen, M., de Ridder, D., Handels, H.: Image processing with neural networks—a review. Pattern Recogn. 35(10), 2279–2301 (2002)CrossRef

Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017)CrossRef

Image retraining Tutorial (n.d.). https://www.tensorflow.org/tutorials/image_retraining#training. Accessed 20 Nov 2017

Inception v3 (2018). https://www.kaggle.com/pytorch/inceptionv3. Accessed 20 Feb 2018

Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS 2012, vol. 1, pp. 1097–1105. Lake Tahoe, CA (2012)

LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRef

Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014), pp. 1717–1724. Columbus, OH (2014). https://doi.org/10.1109/CVPR.2014.222

Pal, N.R., Pal, S.K.: A review on image segmentation techniques. Pattern Recogn. 26(9), 1277–1294 (1993)CrossRef

Rocchini, C., Cignoni, P., Montani, C., Pingi, P., Scopigno, R.: A low cost 3D scanner based on structured light. Comput. Graph. Forum 20(3), 299–308 (2001)CrossRef

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Bernstein, M.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRef

Sharif Razavian, A., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition, In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR-W 2014), pp. 806–813. Columbus, OH (2014). https://doi.org/10.1109/CVPRW.2014.131

Shin, H.-C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Summers, R.M.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285–1298 (2016)CrossRef

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv Preprint arXiv:1409.1556 (2014)

Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V, Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), pp. 1–9. Boston, MA (2015)

Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), pp. 2818–2826. Las Vegas, NV (2016)

Vasey, L., Maxwell, I., Pigram, D.: Adaptive part variation. In: McGee, W., Ponce de Leon, M. (eds.) Robotic Fabrication in Architecture, Art and Design 2014, pp. 291–304. Springer International Publishing Switzerland (2014)

Zhang, L., Curless, B., Seitz, S.M.: Rapid shape acquisition using color structured light and multi-pass dynamic programming. In: Proceedings of First International Symposium on 3D Data Processing Visualization and Transmission (3DPVT 2002), pp. 24–36. Padova, Italy (2002)

Title: Image Classification for Robotic Plastering with Convolutional Neural Network
Authors: Joshua Bard
Ardavan Bidgoli
Wei Wei Chi
Publisher: Springer International Publishing
Book: Robotic Fabrication in Architecture, Art and Design 2018
Print ISBN: 978-3-319-92293-5

Electronic ISBN: 978-3-319-92294-2

Copyright Year: 2019
DOI: https://doi.org/10.1007/978-3-319-92294-2_1

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"