Skip to main content

2020 | OriginalPaper | Buchkapitel

Learning Guided Electron Microscopy with Active Acquisition

verfasst von : Lu Mi, Hao Wang, Yaron Meirovitch, Richard Schalek, Srinivas C. Turaga, Jeff W. Lichtman, Aravinthan D. T. Samuel, Nir Shavit

Erschienen in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2020

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Single-beam scanning electron microscopes (SEM) are widely used to acquire massive datasets for biomedical study, material analysis, and fabrication inspection. Datasets are typically acquired with uniform acquisition: applying the electron beam with the same power and duration to all image pixels, even if there is great variety in the pixels’ importance for eventual use. Many SEMs are now able to move the beam to any pixel in the field of view without delay, enabling them, in principle, to invest their time budget more effectively with non-uniform imaging.
In this paper, we show how to use deep learning to accelerate and optimize single-beam SEM acquisition of images. Our algorithm rapidly collects an information-lossy image (e.g. low resolution) and then applies a novel learning method to identify a small subset of pixels to be collected at higher resolution based on a trade-off between the saliency and spatial diversity. We demonstrate the efficacy of this novel technique for active acquisition by speeding up the task of collecting connectomic datasets for neurobiology by up to an order of magnitude. Code is available at https://​github.​com/​lumi9587/​learning-guided-SEM.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Anderson, H.S., Ilic-Helms, J., Rohrer, B., Wheeler, J., Larson, K.: Sparse imaging for fast electron microscopy. In: Computational Imaging XI, vol. 8657, p. 86570C. International Society for Optics and Photonics (2013) Anderson, H.S., Ilic-Helms, J., Rohrer, B., Wheeler, J., Larson, K.: Sparse imaging for fast electron microscopy. In: Computational Imaging XI, vol. 8657, p. 86570C. International Society for Optics and Photonics (2013)
2.
Zurück zum Zitat Buchholz, T.O., Krull, A., Shahidi, R., Pigino, G., Jékely, G., Jug, F.: Content-aware image restoration for electron microscopy. Meth. Cell Biol. 152, 277–289 (2019)CrossRef Buchholz, T.O., Krull, A., Shahidi, R., Pigino, G., Jékely, G., Jug, F.: Content-aware image restoration for electron microscopy. Meth. Cell Biol. 152, 277–289 (2019)CrossRef
3.
Zurück zum Zitat Dahmen, T.: Feature adaptive sampling for scanning electron microscopy. Sci. Rep. 6, 25350 (2016)CrossRef Dahmen, T.: Feature adaptive sampling for scanning electron microscopy. Sci. Rep. 6, 25350 (2016)CrossRef
4.
Zurück zum Zitat Eberle, A., Mikula, S., Schalek, R., Lichtman, J., Tate, M.K., Zeidler, D.: High-resolution, high-throughput imaging with a multibeam scanning electron microscope. J. Micros. 259(2), 114–120 (2015)CrossRef Eberle, A., Mikula, S., Schalek, R., Lichtman, J., Tate, M.K., Zeidler, D.: High-resolution, high-throughput imaging with a multibeam scanning electron microscope. J. Micros. 259(2), 114–120 (2015)CrossRef
5.
Zurück zum Zitat Eldar, Y.C., Kutyniok, G.: Compressed Sensing: Theory and Applications. Cambridge University Press, Cambridge (2012)CrossRef Eldar, Y.C., Kutyniok, G.: Compressed Sensing: Theory and Applications. Cambridge University Press, Cambridge (2012)CrossRef
6.
Zurück zum Zitat Fang, L., et al.: Deep learning-based point-scanning super-resolution imaging. bioRxiv, p. 740548 (2019) Fang, L., et al.: Deep learning-based point-scanning super-resolution imaging. bioRxiv, p. 740548 (2019)
7.
Zurück zum Zitat Flegler, S.L., Flegler, S.L.: Scanning & Transmission Electron Microscopy. Oxford University Press, Oxford (1997) Flegler, S.L., Flegler, S.L.: Scanning & Transmission Electron Microscopy. Oxford University Press, Oxford (1997)
8.
Zurück zum Zitat Gan, L.: Block compressed sensing of natural images. In: 2007 15th International Conference on Digital Signal Processing, pp. 403–406. IEEE (2007) Gan, L.: Block compressed sensing of natural images. In: 2007 15th International Conference on Digital Signal Processing, pp. 403–406. IEEE (2007)
9.
Zurück zum Zitat Helmstaedter, M., Briggman, K.L., Denk, W.: High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nat. Neurosci. 14(8), 1081–1088 (2011)CrossRef Helmstaedter, M., Briggman, K.L., Denk, W.: High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nat. Neurosci. 14(8), 1081–1088 (2011)CrossRef
11.
Zurück zum Zitat Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017) Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
12.
Zurück zum Zitat Januszewski, M.: High-precision automated reconstruction of neurons with flood-filling networks. Nat. Meth. 15, 605–610 (2018)CrossRef Januszewski, M.: High-precision automated reconstruction of neurons with flood-filling networks. Nat. Meth. 15, 605–610 (2018)CrossRef
13.
Zurück zum Zitat Jarrell, T.A.: The connectome of a decision-making neural network. Science 337(6093), 437–444 (2012)CrossRef Jarrell, T.A.: The connectome of a decision-making neural network. Science 337(6093), 437–444 (2012)CrossRef
14.
Zurück zum Zitat Kasthuri, N., et al.: Saturated reconstruction of a volume of neocortex. Cell 162(3), 648–661 (2015)CrossRef Kasthuri, N., et al.: Saturated reconstruction of a volume of neocortex. Cell 162(3), 648–661 (2015)CrossRef
15.
Zurück zum Zitat Kulesza, A., Taskar, B., et al.: Determinantal point processes for machine learning. Found. Trends Mach. Learn. 5(2–3), 123–286 (2012)CrossRef Kulesza, A., Taskar, B., et al.: Determinantal point processes for machine learning. Found. Trends Mach. Learn. 5(2–3), 123–286 (2012)CrossRef
16.
Zurück zum Zitat Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017) Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)
17.
Zurück zum Zitat Lichtman, J.W., Pfister, H., Shavit, N.: The big data challenges of connectomics. Nat. Neurosci. 17(11), 1448–1454 (2014)CrossRef Lichtman, J.W., Pfister, H., Shavit, N.: The big data challenges of connectomics. Nat. Neurosci. 17(11), 1448–1454 (2014)CrossRef
18.
Zurück zum Zitat Meirovitch, Y., Mi, L., Saribekyan, H., Matveev, A., Rolnick, D., Shavit, N.: Cross-classification clustering: an efficient multi-object tracking technique for 3-D instance segmentation in connectomics. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8425–8435 (2019) Meirovitch, Y., Mi, L., Saribekyan, H., Matveev, A., Rolnick, D., Shavit, N.: Cross-classification clustering: an efficient multi-object tracking technique for 3-D instance segmentation in connectomics. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8425–8435 (2019)
19.
Zurück zum Zitat Mi, L., Wang, H., Tian, Y., Shavit, N.: Training-free uncertainty estimation for neural networks. arXiv preprint arXiv:1910.04858 (2019) Mi, L., Wang, H., Tian, Y., Shavit, N.: Training-free uncertainty estimation for neural networks. arXiv preprint arXiv:​1910.​04858 (2019)
20.
Zurück zum Zitat Mohammed, A.: Scanning electron microscopy (SEM): a review (2018) Mohammed, A.: Scanning electron microscopy (SEM): a review (2018)
21.
Zurück zum Zitat Newell, T., Tillotson, B., Pearl, H., Miller, A.: Detection of electrical defects with semvision in semiconductor production mode manufacturing. In: 2016 27th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), pp. 151–156. IEEE (2016) Newell, T., Tillotson, B., Pearl, H., Miller, A.: Detection of electrical defects with semvision in semiconductor production mode manufacturing. In: 2016 27th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), pp. 151–156. IEEE (2016)
22.
Zurück zum Zitat Pandey, K., Setua, D., Mathur, G.: Material behaviour: fracture topography of rubber surfaces: an SEM study. Polym. Testing 22(3), 353–359 (2003)CrossRef Pandey, K., Setua, D., Mathur, G.: Material behaviour: fracture topography of rubber surfaces: an SEM study. Polym. Testing 22(3), 353–359 (2003)CrossRef
23.
Zurück zum Zitat Potocek, P., Trampert, P., Peemen, M., Schoenmakers, R., Dahmen, T.: Sparse scanning electron microscopy data acquisition and deep neural networks for automated segmentation in connectomics. Microsc. Microanal. 26, 403–412 (2020)CrossRef Potocek, P., Trampert, P., Peemen, M., Schoenmakers, R., Dahmen, T.: Sparse scanning electron microscopy data acquisition and deep neural networks for automated segmentation in connectomics. Microsc. Microanal. 26, 403–412 (2020)CrossRef
25.
Zurück zum Zitat Thorpe, S., Fize, D., Marlot, C.: Speed of processing in the human visual system. Nature 381(6582), 520 (1996)CrossRef Thorpe, S., Fize, D., Marlot, C.: Speed of processing in the human visual system. Nature 381(6582), 520 (1996)CrossRef
26.
Zurück zum Zitat Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T.: Test-time augmentation with uncertainty estimation for deep learning-based medical image segmentation. arXiv preprint arXiv:1807.07356 (2018) Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T.: Test-time augmentation with uncertainty estimation for deep learning-based medical image segmentation. arXiv preprint arXiv:​1807.​07356 (2018)
27.
Zurück zum Zitat Wang, H., et al.: Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Meth. 16, 103–110 (2019)CrossRef Wang, H., et al.: Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Meth. 16, 103–110 (2019)CrossRef
28.
Zurück zum Zitat Weigert, M., et al.: Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Meth. 15(12), 1090 (2018)CrossRef Weigert, M., et al.: Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Meth. 15(12), 1090 (2018)CrossRef
29.
Zurück zum Zitat Yan, G., et al.: Network control principles predict neuron function in the caenorhabditis elegans connectome. Nature 550(7677), 519 (2017)CrossRef Yan, G., et al.: Network control principles predict neuron function in the caenorhabditis elegans connectome. Nature 550(7677), 519 (2017)CrossRef
Metadaten
Titel
Learning Guided Electron Microscopy with Active Acquisition
verfasst von
Lu Mi
Hao Wang
Yaron Meirovitch
Richard Schalek
Srinivas C. Turaga
Jeff W. Lichtman
Aravinthan D. T. Samuel
Nir Shavit
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-59722-1_8

Premium Partner