Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Metallurgical and Materials Transactions A
Published in: Metallurgical and Materials Transactions A 12/2020

29-09-2020 | Metallurgical and Materials Transactions 50th Anniversary Collection

Overview: Computer Vision and Machine Learning for Microstructural Characterization and Analysis

Authors: Elizabeth A. Holm, Ryan Cohn, Nan Gao, Andrew R. Kitahara, Thomas P. Matson, Bo Lei, Srujana Rao Yarasi

Published in: Metallurgical and Materials Transactions A | Issue 12/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Microstructural characterization and analysis is the foundation of microstructural science, connecting materials structure to composition, process history, and properties. Microstructural quantification traditionally involves a human deciding what to measure and then devising a method for doing so. However, recent advances in computer vision (CV) and machine learning (ML) offer new approaches for extracting information from microstructural images. This overview surveys CV methods for numerically encoding the visual information contained in a microstructural image using either feature-based representations or convolutional neural network (CNN) layers, which then provides input to supervised or unsupervised ML algorithms that find associations and trends in the high-dimensional image representation. CV/ML systems for microstructural characterization and analysis span the taxonomy of image analysis tasks, including image classification, semantic segmentation, object detection, and instance segmentation. These tools enable new approaches to microstructural analysis, including the development of new, rich visual metrics and the discovery of processing-microstructure-property relationships.
next article Design and Tailoring of Alloys for Additive Manufacturing

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Appendix
Available only for authorised users
Literature
1.
C. S. Smith: A History of Metallography: The Development of Ideas on the Structure of Metals before 1890. (MIT Press, Cambridge, MA, 1988), pp. 1-314.
2.
H. C. Sorby, J. Iron and Steel Institute 1887, vol. 37, pp. 255-288.
3.
G. F. Vander-Voort, Standardization News 1991, vol. 19, pp. 42-47.
4.
5.
B.L. DeCost, M. D. Hecht, T. Francis, Y. N. Picard, B. A. Webler and E. A. Holm, Integr. Mater. Manuf. Innov. 2017, vol. 6, pp. 197-205.
6.
G. F. Vander-Voort: Metallography: Principles and Practice (ASM International, Metals Park, OH, 1999), pp. 1-752.
7.
E. E. Underwood: Quantitative Stereology. (Addison-Wesley, New York, 1970), pp. 1-274.
8.
D. Dingley, J. Microscopy 2004, vol. 213, pp. 214-224.
9.
H. F. Poulsen, S. F. Nielsen, E. M. Lauridsen, S. Schmidt, R. M. Suter, U. Lienert, L. Margulies, T. Lorentzen and D. Juul-Jensen, Journal of Applied Crystallography 2001, vol. 34, pp. 751-756.
10.
E. O. Hall, Phys. Soc. Lond. 1951, vol. 64, pp. 747-753.
11.
N. J. Petch, J. Iron Steel Inst. 1953, vol. 174, pp. 25-28.
12.
Min-nan Feng, Yu-cong Wang, Hao Wang, Guo-quan Liu and Wei-hua Xue, International Journal of Minerals, Metallurgy, and Materials 2017, vol. 24, pp. 257-263.
13.
J. J. Friel, E. B. Prestridge, and F. Glazer, in MiCon 90: Advances in Video Technology for Microstructural Control, G. Vander Voort, ed., ASTM International: Philadelphia, PA, 1991, pp. 170–184.
14.
Richard Szeliski: Computer vision: algorithms and applications. (Springer, New York, 2010), pp. 1-812.
15.
Satoshi Yoshinaga, Atsushi Shimada and Rin-ichiro Taniguchi, Procedia - Social and Behavioral Sciences 2010, vol. 2, pp. 143-152.
16.
A. P. Carleer, Olivier Debeir and Eléonore Wolff, Photogrammetric Engineering & Remote Sensing 2005, vol. 71, pp. 1285-1294.
17.
P. Flach: Machine Learning: The Art and Science of Algorithms that Make Sense of Data. (Cambridge University Press, Cambridge, UK, 2012), pp. 1-409.
18.
O. B. Abouelatta, J.Amer. Sci. 2013, vol. 9, pp. 213-223.
19.
A. Cecen, T. Fast, E. C. Kumbur and S. R. Kalidindi, Journal of Power Sources 2014, vol. 245, pp. 144-153.
20.
Aritra Chowdhury, Elizabeth Kautz, B¸lent Yener and Daniel Lewis, Comput. Mater. Sci. 2016, vol. 123, pp. 176-187.
21.
V.H.C. De-Albuquerque, P.C. Cortez, A.R. De-Alexandria, J.M.R.S. Tavares, Nondestruct. Test. Eval. 2008, vol. 23, pp. 273-283.
22.
S. R. Kalidindi and D. T. Fullwood, Jom 2007, vol. 59, pp. 26-31.
23.
S. R. Kalidindi, S. R. Niezgoda and A. A. Salem, Jom 2011, vol. 63, pp. 34-41.
24.
A. C. Lewis, C. Suh, M. Stukowski, A. B. Geltmacher, G. Spanos and K. Rajan, JOM 2006, vol. 58, pp. 52-56.
25.
S. R. Niezgoda, D. T. Fullwood and S. R. Kalidindi, Acta Materialia 2008, vol. 56, pp. 5285-5292.
26.
S. R. Niezgoda and S. R. Kalidindi, CMC-Comput. Mat. Contin. 2009, vol. 14, pp. 79-97.
27.
S.R. Niezgoda, S.R. Kalidindi, X. Hu, G.A. Cingara, D.S. Wilkinson, M. Jain, P. Wu, R.K. Mishra, M. Arafin, J. Szpunar, Comput. Mater. Continua, 2010, vol. 14, pp. 79-98.
28.
S.R. Niezgoda, A.K. Kanjarla, S.R. Kalidindi, Integr. Mater. Manuf. Innov. 2013, vol. 2, pp. 1-27.
29.
S. R. Niezgoda, Y. C. Yabansu and S. R. Kalidindi, Acta Materialia 2011, vol. 59, pp. 6387-6400.
30.
G. Saheli, H. Garmestani and B. L. Adams, J. Comput-Aided Mater. Des. 2004, vol. 11, pp. 103-115.
31.
J. P. Simmons, P. Chuang, M. Comer, J. E. Spowart, M. D. Uchic and M. De-Graef, Modell. Simul. Mater. Sci. Eng. 2009, vol. 17, pp. 0250021-02500222.
32.
A. Velichko, C. Holzapfel, A. Siefers, K. Schladitz and F. Mucklich, Acta Materialia 2008, vol. 56, pp. 1981-1990.
33.
B.L. Decost, Materials Science and Engineering (Carnegie Mellon University, Pittsburgh, PA, 2016), pp. 1-169.
34.
B.L. DeCost, T. Francis and E.A. Holm, Acta Mater. 2017, vol. 133, pp. 30-40.
35.
B.L. DeCost and E. Holm, Comput. Mater. Sci. 2017, vol. 126, pp. 438-445.
36.
B.L. DeCost and E. Holm, in Statistical Methods for Materials Science: Data Analytics in Microstructure Characterization, J. Simmons, C. Bouman, L. Drummy, and M. de Graef, ed., CRC Press, Boca Raton, FL, 2017, pp. 73–93.
37.
Brian L. DeCost and Elizabeth A. Holm, Comput. Mater. Sci. 2015, vol. 110, pp. 126-133.
38.
B.L. DeCost, H. Jain, E. Holm, A. Rollett, JOM 2017, vol. 69, pp. 456-465.
39.
Dana H Ballard, Pattern recognition 1981, vol. 13, pp. 111-122.
40.
41.
X. Chen, A. Shrivastava, and A. Gupta, 2013 IEEE International Conference on Computer Vision (ICCV), 2013, pp. 1409–16.
42.
Mary Comer, Charles A. Bouman, Marc De Graef and Jeff P. Simmons, Jom 2011, vol. 63, pp. 55-57.
43.
N. Dalal and B. Triggs, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (CVPR), (2005), pp. 886–93.
44.
P.F. Felzenszwalb, R.B. Girshick, D. McAllester, and D. Ramanan, IEEE Trans. Pattern Anal. Mach. Intell., 2010, vol. 32, pp. 1627-1645.
45.
46.
C. Harris and M. Stephens, Proceedings of the Fourth Alvey Vision Conference (Machester, UK), 1988, pp. 147–151.
47.
Yu-Gang Jiang, Jun Yang, Chong-Wah Ngo and Alexander G. Hauptmann, Ieee Transactions on Multimedia 2010, vol. 12, pp. 42-53.
48.
T.-Y. Lin, A. RoyChowdhury, and S. Maji, Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1449–57.
49.
D.G. Lowe, Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999, pp. 1150–57.
50.
D.G. Lowe, International Journal of Computer Vision, 2004, vol. 60, pp. 91-110.
51.
Aude Oliva and Antonio Torralba, Progress in brain research 2006, vol. 155, pp. 23-36.
52.
H. Peng, F. Long and C. Ding, Ieee Transactions on Pattern Analysis and Machine Intelligence 2005, vol. 27, pp. 1226-1238.
53.
Florent Perronnin, Ieee Transactions on Pattern Analysis and Machine Intelligence 2008, vol. 30, pp. 1243-1256.
54.
Pedro Quelhas, Florent Monay, Jean-Marc Odobez, Daniel Gatica-Perez and Tinne Tuytelaars, Ieee Transactions on Pattern Analysis and Machine Intelligence 2007, vol. 29, pp. 1575-1589.
55.
F.S. Khan, R.M. Anwer, J. van de Weijer, A.D. Bagdanov, M. Vanrell, and A.M. Lopez, 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 3306–13.
56.
J.C. van Gemert, C.J. Veenman, A.W.M. Smeulders, and J.-M. Geusebroek, IEEE Trans. Pattern Anal. Mach. Intell. 2010, vol. 32, pp. 1271-1283.
57.
Jianguo Zhang, Marcin Marszalek, Svetlana Lazebnik and Cordelia Schmid, International journal of computer vision 2007, vol. 73, pp. 213-238.
58.
Jing Li and Nigel M. Allinson, Neurocomputing 2008, vol. 71, pp. 1771-1787.
59.
G. Csurka, C. R. Dance, L. Fan, J. Willamowski, and C. Bray, Workshop on Statistical Learning in Computer Vision, ECCV, 2004, vol. 1, pp. 1–16.
60.
J. Sivic and A. Zisserman, Proceedings Ninth IEEE International Conference on Computer Vision, 2003, pp. 1470–77.
61.
Thiago S. Guzella and Walmir M. Caminhas, Expert Systems with Applications 2009, vol. 36, pp. 10206-10222.
62.
H. Jégou, M. Douze, C. Schmid, and P. Pérez, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, pp. 3304–11.
63.
R.M. Cichy, A. Khosla, D. Pantazis, A. Torralba, and A. Oliva, Sci. Rep. 2016, vol. 6, pp. 277551–2775513.
64.
M.D. Zeiler, G.W. Taylor, and R. Fergus, 2011 International Conference on Computer Vision, 2011, pp. 2018–25.
65.
66.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, Curran Associates Inc., Lake Tahoe, Nevada, 2012, pp. 1097–1105.
67.
A. Jalalian, S.B.T. Mashohor, H.R. Mahmud, M. Iqbal, B. Saripan, A. Rahman, B. Ramli, and B. Karasfi, Clin. Imaging 2013, vol. 37, pp. 420-426.
68.
I. Goodfellow, Y. Bengio and A. Courville: Deep Learning (MIT Press, Cambridge, MA, 2016), pp. 1-800.
69.
Yann LeCun, Yoshua Bengio and Geoffrey Hinton, Nature 2015, vol. 521, pp. 436-444.
70.
J. Schmidhuber, Neural Networks 2015, vol. 61, pp. 85-117.
71.
Yanming Guo, Yu Liu, Ard Oerlemans, Songyang Lao, Song Wu and Michael S. Lew, Neurocomputing 2015, vol. 187, pp. 27-48.
72.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. Berg, and L. Fei-Fei, Int. J. Comput. Vis., 2015, vol. 115, pp. 211-252.
73.
Hervé Abdi and Lynne J. Williams, Wiley Interdisciplinary Reviews: Computational Statistics 2010, vol. 2, pp. 433-459.
74.
75.
Bharath Hariharan, Jitendra Malik and Deva Ramanan, In Computer Vision–ECCV 2012, (Springer: New York, 2012), pp. 459-472.
76.
D.P. Kingma and M. Welling, arXiv:1312.6114 [stat.ML], 2013.
77.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farly, S. Ozair, A. Courville, and Y. Bengio, Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014), 2014, pp. 2672–80.
78.
Z. Yang, X. Li, L.C. Brinson, A.N. Choudhary, W. Chen and A. Agrawal, J. Mech. Des. 2018, vol. 140, 110301.
79.
S. Chun, S. Roy, Y. T. Nguyen, J.B. Choi, H. S. Udaykumar and S.S. Baek, Sci. Rep. 2020, vol. 10, pp. 13307-13307.
80.
A. Gayon-Lombardo, L. Mosser, N.P. Brandon, and S.J. Cooper, NPJ Comput. Mater., 2020, vol. 6, p. 82.
81.
Honggang Chen, Xiaohai He, Qizhi Teng, Raymond E. Sheriff, Junxi Feng and Shuhua Xiong, Physical Review E 2020, vol. 101, p. 023305.
82.
T. M. Mitchell: Machine Learning. (McGraw-Hill, New York, 1997), pp. 1-414.
83.
C. Cortes and V. Vapnik, Machine Learning 1995, vol. 20, pp. 273-297.
84.
Bernhard Scholkopf and Alexander J. Smola: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. (MIT Press, Cambridge, MA, 2001), pp. 1-632.
85.
Andy Liaw and Matthew Wiener, R News 2002, vol. 2/3, pp. 18-22.
86.
A. Mangal and E. A. Holm, Integrated Materials and Manufacturing Innovation 2018, vol. 7, pp. 87-95.
87.
S. Lloyd, IEEE Transactions on Information Theory 1982, vol. 28, pp. 129-137.
88.
L. van der Maaten, E. Postma, and H. Herik, J. Mach. Learn. Res. JMLR 2007, vol. 10, 301-348.
89.
G. Hinton and S. Roweis, Proceedings of Advances in Neural Information Processing Systems, 2002, vol. 15, pp. 833–841.
90.
L. van der Maaten and G. Hinton, J. Mach. Learn. Res, 2008, vol. 9, pp. 2579-2605.
91.
92.
Connor Shorten and Taghi M. Khoshgoftaar, Journal of Big Data 2019, vol. 6, p. 60.
93.
A. R. Kitahara and E. A. Holm, Integrated Materials and Manufacturing Innovation 2018, vol. 7, pp. 148-156.
94.
J. Masci, U. Meier, D. Ciresan, J. Schmidhuber, and G. Fricout, The 2012 International Joint Conference on Neural Networks (IJCNN), 2012, pp. 1–6.
95.
W. Li, K.G. Field, and D. Morgan, NPJ Comput. Mater., 2018, vol. 4, p. 36.
96.
S.M. Azimi, D. Britz, M. Engstler, M. Fritz, and F. Mücklich, Sci. Rep., 2018, vol. 8, p. 2128.
97.
J. Madsen, P. Liu, J. Kling, J.B. Wagner, T.W. Hansen, O. Winther, and J. Schiøtz, Adv. Theory Simul., 2018, vol. 1, p. 1800037.
98.
Carl Kusche, Tom Reclik, Martina Freund, Talal Al-Samman, Ulrich Kerzel and Sandra Korte-Kerzel, PLOS ONE 2019, vol. 14, p. e0216493.
99.
100.
Ruho Kondo, Shunsuke Yamakawa, Yumi Masuoka, Shin Tajima and Ryoji Asahi, Acta Materialia 2017, vol. 141, pp. 29-38.
101.
S. Lathuiliére, P. Mesejo, X. Alameda-Pineda and R. Horaud, IEEE Trans. Pattern Anal. Mach. Intell., 2019, 36, pp. 1.
102.
ASTM: E112-13 Standard Test Methods for Determining Average Grain Size, ASTM International, West Conshohocken, PA, 2013, pp. 1–28.
103.
H. V. Atkinson and G. Shi, Progress in Materials Science 2003, vol. 48, pp. 457-520.
104.
C.A. Schneider, W.S. Rasband and K.W. Eliceiri, Nature Methods 2012, vol. 9, pp. 671-675.
105.
Alberto Garcia-Garcia, Sergio Orts-Escolano, Sergiu Oprea, Victor Villena-Martinez, Pablo Martinez-Gonzalez and Jose Garcia-Rodriguez, Applied Soft Computing 2018, vol. 70, pp. 41-65.
106.
B.L. DeCost, T. Francis, and E.A. Holm, Microsc. Microanal., 2019, vol. 25, pp. 21-29.
107.
Tiberiu Stan, Zachary Thompson and Peter Voorhees, Materials Characterization 2020, vol. 160, p. 110119.
108.
S. Sulzer, Z. Li, S. Zaefferer, M. H. Haghighat, A. Wilkinson, D. Raabe, and R. Reed, Acta Mater., 2020, vol. 185, pp. 13-27.
109.
O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation. Springer, Cham, 2015, pp. 234-241.
110.
J. Cousty, G. Bertrand, L. Najman and M. Couprie, IEEE Transactions on Pattern Analysis and Machine Intelligence 2009, vol. 31, pp. 1362-1374.
111.
Christophe Fiorio and Jens Gustedt, Theoretical Computer Science 1996, vol. 154, pp. 165-181.
112.
Kesheng Wu, Ekow Otoo and Kenji Suzuki, Pattern Analysis and Applications 2009, vol. 12, pp. 117-135.
113.
Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu and Matti Pietikäinen, International Journal of Computer Vision 2020, vol. 128, pp. 261-318.
114.
S. Ren, K. He, R. Girshick and J. Sun, IEEE Transactions on Pattern Analysis and Machine Intelligence 2017, vol. 39, pp. 1137-1149.
115.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. Lawrence Zitnick, European Conference on Computer vision ECCV 2014, 2014, pp. 740–55.
116.
117.
K. He, G. Gkioxari, P. Dollár, and R. Girshick, 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2980–88.
118.
A. Strondl, O. Lyckfeldt, H. Brodin and U. Ackelid, JOM 2015, vol. 67, pp. 549-554.
119.
Nicholas Lubbers, Turab Lookman and Kipton Barros, Physical Review E 2017, vol. 96, p. 052111.
120.
X. Han, H. Laga and M. Bennamoun, IEEE Trans. Pattern Anal. Mach. Intell., 2019, 40, 1224–44.
121.
E.E. Underwood, in Microstructural Analysis: Tools and Techniques, J.L. McCall and W.M. Mueller, ed., Springer, Boston, MA, 1973, pp. 35–66.
122.
R. Noraas, N. Somanath, M. Giering, and O.O. Olusegun, AIAA Scitech 2019 Forum.
123.
A. Mangal and E. A. Holm, International Journal of Plasticity 2018, vol. 111, pp. 122-134.
124.
Ankita Mangal and Elizabeth A. Holm, International Journal of Plasticity 2019, vol. 114, pp. 1-14.
125.
C. Kantzos, J. Lao and A. Rollett, Materials Characterization 2019, vol. 158, p. 109961.
126.
H. Xu, R. Liu, A. Choudhary, and W. Chen, J. Mech. Des., 2015, vol. 137, 050301.
127.
128.
R.C. Fong and A. Vedaldi, 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 3449–57.
Metadata
Title
Overview: Computer Vision and Machine Learning for Microstructural Characterization and Analysis
Authors
Elizabeth A. Holm
Ryan Cohn
Nan Gao
Andrew R. Kitahara
Thomas P. Matson
Bo Lei
Srujana Rao Yarasi
Publication date
29-09-2020
Publisher
Springer US
Published in
Metallurgical and Materials Transactions A / Issue 12/2020
Print ISSN: 1073-5623
Electronic ISSN: 1543-1940
DOI
https://doi.org/10.1007/s11661-020-06008-4

Other articles of this Issue 12/2020

Metallurgical and Materials Transactions A 12/2020 Go to the issue

OriginalPaper

A Study of the Microstructure, Texture and Tensile Properties of 1060/Al-TiC/1060 Sandwich Composites Prepared by Hot-Roll Bonding

OriginalPaper

Hot Tearing Behavior in Double Ternary Eutectic Alloy System: Mg-Ce-Al Alloys

Topical Collection: Biodegradable Materials for Medical Applications II

Fabrication of a Porous and Formable Ceramic Composite Bone Tissue Scaffold at Ambient Temperature

OriginalPaper

Validation of Novel Geometrically Necessary Dislocations Calculation Model Using Nanoindentation of the Metal Matrix Nanocomposite

OriginalPaper

Influence of Hot Isostatic Pressing on the Hot Ductility of Cast Alloy 718: The Effect of Niobium and Minor Elements on the Liquation Mechanism

OriginalPaper

Evolution of Sliver Defect in Ni-Based Single Crystal Superalloy

Premium Partners

    Image Credits