Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
ATZ-Fachkongress

Chassis.tech plus 2024


4 Kongresse in einer Veranstaltung

Treffen Sie in München die Fachleute von Chassis, Lenkung, Bremsen sowie Reifen/Räder.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Weakly-Supervised Crowd Counting Learns from Sorting Rather Than Locations Kapitel lesen Erstes Kapitel lesen

2020 | OriginalPaper | Buchkapitel

Image Classification in the Dark Using Quanta Image Sensors

verfasst von : Abhiram Gnanasambandam, Stanley H. Chan

Erschienen in: Computer Vision – ECCV 2020

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

State-of-the-art image classifiers are trained and tested using well-illuminated images. These images are typically captured by CMOS image sensors with at least tens of photons per pixel. However, in dark environments when the photon flux is low, image classification becomes difficult because the measured signal is suppressed by noise. In this paper, we present a new low-light image classification solution using Quanta Image Sensors (QIS). QIS are a new type of image sensors that possess photon-counting ability without compromising on pixel size and spatial resolution. Numerous studies over the past decade have demonstrated the feasibility of QIS for low-light imaging, but their usage for image classification has not been studied. This paper fills the gap by presenting a student-teacher learning scheme which allows us to classify the noisy QIS raw data. We show that with student-teacher learning, we can achieve image classification at a photon level of one photon per pixel or lower. Experimental results verify the effectiveness of the proposed method compared to existing solutions.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel REMIND Your Neural Network to Prevent Catastrophic Forgetting
Nächstes Kapitel n-Reference Transfer Learning for Saliency Prediction
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Ba, J., Caruana, R.: Do deep nets really need to be deep? In: NeurIPS (2014)
2.
Callenberg, C., Lyons, A., den Brok, D., Henderson, R., Hullin, M.B., Faccio, D.: EMCCD-SPAD camera data fusion for high spatial resolution time-of-flight imaging. In: Computational Optical Sensing and Imaging (2019)
3.
Chan, S.H., Elgendy, O.A., Wang, X.: Images from bits: non-iterative image reconstruction for quanta image sensors. Sensors 16(11), 1961 (2016)
4.
Chan, S.H., Lu, Y.M.: Efficient image reconstruction for gigapixel quantum image sensors. In: IEEE Global Conference on Signal and Information Processing (2014)
5.
Chan, S.H., Wang, X., Elgendy, O.A.: Plug-and-play ADMM for image restoration: fixed-point convergence and applications. IEEE Trans. Comput. Imaging 3(1), 84–98 (2016)MathSciNet
6.
Chandramouli, P., Burri, S., Bruschini, C., Charbon, E., Kolb, A.: A bit too much? High speed imaging from sparse photon counts. In: ICCP (2019)
7.
Chen, B., Perona, P.: Seeing into darkness: scotopic visual recognition. In: CVPR (2017)
8.
Chen, C., Chen, Q., Do, M.N., Koltun, V.: Seeing motion in the dark. In: ICCV (2019)
9.
Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: CVPR (2018)
10.
Chen, G., Li, Y., Srihari, S.N.: Joint visual denoising and classification using deep learning. In: ICIP (2016)
11.
Choi, J.H., Elgendy, O.A., Chan, S.H.: Image reconstruction for quanta image sensors using deep neural networks. In: ICASSP (2018)
12.
Davy, A., Ehret, T., Morel, J.M., Arias, P., Facciolo, G.: A non-local CNN for video denoising. In: ICIP (2019)
13.
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)
14.
Diamond, S., Sitzmann, V., Boyd, S., Wetzstein, G., Heide, F.: Dirty pixels: optimizing image classification architectures for raw sensor data. arXiv preprint arXiv:​1701.​06487 (2017)
15.
16.
Elgendy, O.A., Chan, S.H.: Optimal threshold design for quanta image sensor. IEEE Trans. Comput. Imaging 4(1), 99–111 (2017)MathSciNet
17.
18.
Fossum, E.R.: Gigapixel digital film sensor (DFS) proposal. In: Nanospace Manipulation of Photons and Electrons for Nanovision Systems (2005)
19.
Fossum, E.R.: Some thoughts on future digital still cameras. In: Image Sensors and Signal Processing for Digital Still Cameras (2006)
20.
Fossum, E.R.: Modeling the performance of single-bit and multi-bit quanta image sensors. IEEE J. Electron Dev. Soc. 1(9), 166–174 (2013)
21.
Fossum, E.R., Ma, J., Masoodian, S., Anzagira, L., Zizza, R.: The quanta image sensor: every photon counts. Sensors 16(8), 1260 (2016)
22.
Fowler, B., McGrath, D., Bartkovjak, P.: Read noise distribution modeling for CMOS image sensors. In: International Image Sensor Workshop (2013)
23.
Fu, Q., Jung, C., Xu, K.: Retinex-based perceptual contrast enhancement in images using luminance adaptation. IEEE Access 6, 61277–61286 (2018)
24.
Gariepy, G., et al.: Single-photon sensitive light-in-fight imaging. Nat. Commun. 6(1), 1–7 (2015)MathSciNet
25.
Gatys, L., Ecker, A., Bethge, M.: A neural algorithm of artistic style. Nat. Commun. (2015)
26.
Gatys, L.A., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: NeurIPS (2015)
27.
Gnanasambandam, A., Elgendy, O., Ma, J., Chan, S.H.: Megapixel photon-counting color imaging using quanta image sensor. Opt. Express 27(12), 17298–17310 (2019)
28.
Gnanasambandam, A., Ma, J., Chan, S.H.: High dynamic range imaging using quanta image sensors. In: International Image Sensors Workshop (2019)
29.
Guo, T., Xu, C., He, S., Shi, B., Xu, C., Tao, D.: Robust student network learning. IEEE Trans. Neural Netw. Learn. Syst. 31(7), 2455–2468 (2020)
30.
Guo, X., Li, Y., Ling, H.: LIME: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2016)MathSciNetMATH
31.
Gupta, A., Ingle, A., Gupta, M.: Asynchronous single-photon 3D imaging. In: ICCV, October 2019
32.
Gyongy, I., Dutton, N., Henderson, R.: Single-photon tracking for high-speed vision. Sensors 18(2), 323 (2018)
33.
Hasinoff, S.W., et al.: Burst photography for high dynamic range and low-light imaging on mobile cameras. ACM Trans. Graph. 35(6), 192 (2016)
34.
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. In: NeurIPS Deep Learning and Representation Learning Workshop (2015)
35.
Hossain, M.T., Teng, S.W., Zhang, D., Lim, S., Lu, G.: Distortion robust image classification using deep convolutional neural network with discrete cosine transform. In: ICIP (2019)
36.
Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:​1704.​04861 (2017)
37.
Hu, Z., Cho, S., Wang, J., Yang, M.H.: Deblurring low-light images with light streaks. In: CVPR (2014)
38.
39.
Joshi, N., Cohen, M.: Seeing Mt. Rainier: lucky imaging for multi-image denoising, sharpening, and haze removal. In: ICCP (2010)
40.
Khosla, A., Jayadevaprakash, N., Yao, B., Li, F.F.: Novel dataset for fine-grained image categorization: stanford dogs. In: CVPR Workshop on Fine-Grained Visual Categorization (2011)
41.
Kokkinos, F., Lefkimmiatis, S.: Iterative residual CNNs for burst photography applications. In: CVPR (2019)
42.
Lindell, D.B., O’Toole, M., Wetzstein, G.: Single-photon 3D imaging with deep sensor fusion. ACM Trans. Graph. 37(4), 1–12 (2018)
43.
44.
Liu, D., Wen, B., Jiao, J., Liu, X., Wang, Z., Huang, T.S.: Connecting image denoising and high-level vision tasks via deep learning. IEEE Trans. Image Process. 29, 3695–3706 (2020)
45.
Liu, Z., Zhou, T., Shen, Z., Kang, B., Darrell, T.: Transferable recognition-aware image processing. arXiv preprint arXiv:​1910.​09185 (2019)
46.
Liu, Z., Yuan, L., Tang, X., Uyttendaele, M., Sun, J.: Fast burst images denoising. ACM Trans. Graph. 33(6), 232 (2014)
47.
Lore, K.G., Akintayo, A., Sarkar, S.: LLNet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recogn. 61, 650–662 (2017)
48.
Ma, J., Fossum, E.: A pump-gate jot device with high conversion gain for a quanta image sensor. IEEE J. Electron Dev. Soc. 3(2), 73–77 (2015)
49.
Ma, J., Masoodian, S., Starkey, D., Fossum, E.R.: Photon-number-resolving megapixel image sensor at room temperature without avalanche gain. Optica 4(12), 1474–1481 (2017)
50.
Ma, S., Gupta, S., Ulku, A.C., Brushini, C., Charbon, E., Gupta, M.: Quanta burst photography. ACM Trans. Graph. (TOG) 39(4) (2020)
51.
Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: CVPR (2015)
52.
Makitalo, M., Foi, A.: Optimal inversion of the Anscombe transformation in low-count Poisson image denoising. IEEE Trans. Image Process. 20(1), 99–109 (2010)MathSciNetMATH
53.
Malm, H., Oskarsson, M., Warrant, E., Clarberg, P., Hasselgren, J., Lejdfors, C.: Adaptive enhancement and noise reduction in very low light-level video. In: ICCV (2007)
54.
Mildenhall, B., Barron, J.T., Chen, J., Sharlet, D., Ng, R., Carroll, R.: Burst denoising with kernel prediction networks. In: CVPR (2018)
55.
O’Toole, M., Heide, F., Lindell, D.B., Zang, K., Diamond, S., Wetzstein, G.: Reconstructing transient images from single-photon sensors. In: CVPR (2017)
56.
Plotz, T., Roth, S.: Benchmarking denoising algorithms with real photographs. In: CVPR (2017)
57.
Remez, T., Litany, O., Giryes, R., Bronstein, A.: Deep convolutional denoising of low-light images. arXiv preprint arXiv:​1701.​01687 (2017)
58.
Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. ICLR (2014)
59.
60.
Talebi, H., Milanfar, P.: Learned perceptual image enhancement. In: ICCP (2018)
61.
Wu, J., Timofte, R., Huang, Z., Van Gool, L.: On the relation between color image denoising and classification. arXiv preprint arXiv:​1704.​01372 (2017)
62.
Xu, J., Li, H., Liang, Z., Zhang, D., Zhang, L.: Real-world noisy image denoising: a new benchmark. arXiv preprint arXiv:​1804.​02603 (2018)
63.
Yang, C., Xie, L., Qiao, S., Yuille, A.: Training deep neural networks in generations: a more tolerant teacher educates better students. In: AAAI Conference on Artificial Intelligence, July 2019
64.
Yang, F., Lu, Y.M., Sbaiz, L., Vetterli, M.: An optimal algorithm for reconstructing images from binary measurements. In: Proceedings of SPIE, vol. 7533, pp. 158–169 (2010)
65.
Yang, F., Lu, Y.M., Sbaiz, L., Vetterli, M.: Bits from photons: Oversampled image acquisition using binary Poisson statistics. IEEE Trans. Image Process. 21(4), 1421–1436 (2011)MathSciNetMATH
66.
Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. In: ICML Deep Learning Workshop (2015)
67.
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)MathSciNetMATH
68.
Zhang, K., Zuo, W., Zhang, L.: FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)MathSciNet
69.
Zhang, Y., Xiang, T., Hospedales, T.M., Lu, H.: Deep mutual learning. In: CVPR (2018)
Metadaten
Titel
Image Classification in the Dark Using Quanta Image Sensors
verfasst von
Abhiram Gnanasambandam
Stanley H. Chan
Copyright-Jahr
2020
Buch
Computer Vision – ECCV 2020

Print ISBN: 978-3-030-58597-6

Electronic ISBN: 978-3-030-58598-3

Copyright-Jahr: 2020

DOI
https://doi.org/10.1007/978-3-030-58598-3_29

Premium Partner

    Bildnachweise