skip to main content
research-article

Spatial-spectral encoded compressive hyperspectral imaging

Published:19 November 2014Publication History
Skip Abstract Section

Abstract

This paper proposes a novel compressive hyperspectral (HS) imaging approach that allows for high-resolution HS images to be captured in a single image. The proposed architecture comprises three key components: spatial-spectral encoded optical camera design, over-complete HS dictionary learning and sparse-constraint computational reconstruction. Our spatial-spectral encoded sampling scheme provides a higher degree of randomness in the measured projections than previous compressive HS imaging approaches; and a robust nonlinear sparse reconstruction method is employed to recover the HS images from the coded projection with higher performance. To exploit the sparsity constraint on the nature HS images for computational reconstruction, an over-complete HS dictionary is learned to represent the HS images in a sparser way than previous representations. We validate the proposed approach on both synthetic and real captured data, and show successful recovery of HS images for both indoor and outdoor scenes. In addition, we demonstrate other applications for the over-complete HS dictionary and sparse coding techniques, including 3D HS images compression and denoising.

Skip Supplemental Material Section

Supplemental Material

References

  1. Aharon, M., Elad, M., and Bruckstein, A. 2006. K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Proc. 54, 11, 4311--4322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Arguello, H., Rueda, H., Wu, Y., Prather, D. W., and Arce, G. R. 2013. Higher-order computational model for coded aperture spectral imaging. Applied Optics 52, 10, D12--D21.Google ScholarGoogle ScholarCross RefCross Ref
  3. August, Y., Vachman, C., Rivenson, Y., and Stern, A. 2013. Compressive hyperspectral imaging by random separable projections in both the spatial and the spectral domains. Applied Optics 52, 10, D46--D54.Google ScholarGoogle ScholarCross RefCross Ref
  4. Basedow, R. W., Carmer, D. C., and Anderson, M. E. 1995. Hydice system: Implementation and performance. In Proc. SPIE, 258--267.Google ScholarGoogle Scholar
  5. Candes, E. J., and Tao, T. 2005. Decoding by linear programming. IEEE Trans. Inform. Theory 51, 12, 4203--4215. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Candes, E. J., Eldar, Y. C., Needell, D., and Randall, P. 2011. Compressed sensing with coherent and redundant dictionaries. Appl. Comput. Harmon. Anal. 31, 1, 59--73.Google ScholarGoogle ScholarCross RefCross Ref
  7. Chakrabarti, A., and Zickler, T. 2011. Statistics of real-world hyperspectral images. In Proc. IEEE CVPR, 193--200. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Chi, C., Yoo, H., and Ben-Ezra, M. 2010. Multi-spectral imaging by optimized wide band illumination. IJCV 86, 2-3, 140--151. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Donoho, D. L., and Huo, X. 2001. Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inform. Theory 47, 7, 2845--2862. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Donoho, D. L. 2006. Compressed sensing. IEEE Trans. Inform. Theory 52, 4, 1289--1306. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Du, H., Tong, X., Cao, X., and Lin, S. 2009. A prism-based system for multispectral video acquisition. In Proc. IEEE ICCV, 175--182.Google ScholarGoogle Scholar
  12. Duarte-Carvajalino, J. M., and Sapiro, G. 2009. Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization. IEEE Trans. Im. Proc. 18, 7, 1395--1408. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Elad, M., and Aharon, M. 2006. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Im. Proc. 15, 12, 3736--3745. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Elad, M. 2007. Optimized projections for compressed sensing. IEEE Trans. Signal Proc. 55, 12, 5695--5702. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Gao, L., Kester, R. T., Hagen, N., and Tkaczyk, T. S. 2010. Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy. Optics Express 18, 14, 14330--14344.Google ScholarGoogle ScholarCross RefCross Ref
  16. Gat, N., Scriven, G., Garman, J., De Li, M., and Zhang, J. 2006. Development of four-dimensional imaging spectrometers (4D-IS). In Proc. SPIE Optics+ Photonics, 63020M--63020M.Google ScholarGoogle Scholar
  17. Gat, N. 2000. Imaging spectroscopy using tunable filters: a review. In Proc. AeroSense, 50--64.Google ScholarGoogle Scholar
  18. Gehm, M., John, R., Brady, D., Willett, R., and Schulz, T. 2007. Single-shot compressive spectral imaging with a dual-disperser architecture. Optics Express 15, 21, 14013--14027.Google ScholarGoogle ScholarCross RefCross Ref
  19. Gorman, A., Fletcher-Holmes, D. W., and Harvey, A. R. 2010. Generalization of the lyot filter and its application to snapshot spectral imaging. Optics Express 18, 6, 5602--5608.Google ScholarGoogle ScholarCross RefCross Ref
  20. Han, S., Sato, I., Okabe, T., and Sato, Y. 2011. Fast spectral reflectance recovery using DLP projector. In Proc. ACCV, 323--335. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., and Nayar, S. K. 2011. Video from a single coded exposure photograph using a learned over-complete dictionary. In Proc. IEEE ICCV, 287--294. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Hullin, M. B., Hanika, J., Ajdin, B., Seidel, H.-P., Kautz, J., and Lensch, H. 2010. Acquisition and analysis of bispectral bidirectional reflectance and reradiation distribution functions. ACM Trans. Graph. (SIGGRAPH) 29, 4, 97. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Johnson, W. R., Wilson, D. W., and Bearman, G. 2006. Spatial-spectral modulating snapshot hyperspectral imager. Applied Optics 45, 9, 1898--1908.Google ScholarGoogle ScholarCross RefCross Ref
  24. Kawakami, R., Wright, J., Tai, Y.-W., Matsushita, Y., Ben-Ezra, M., and Ikeuchi, K. 2011. High-resolution hyperspectral imaging via matrix factorization. In Proc. IEEE CVPR, 2329--2336. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Kim, M. H., Harvey, T. A., Kittle, D. S., Rushmeier, H., Dorsey, J., Prum, R. O., and Brady, D. J. 2012. 3D imaging spectroscopy for measuring hyperspectral patterns on solid objects. ACM Trans. Graph. (SIGGRAPH) 31, 4, 38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Kittle, D., Choi, K., Wagadarikar, A., and Brady, D. J. 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Applied Optics 49, 36, 6824--6833.Google ScholarGoogle ScholarCross RefCross Ref
  27. Lin, X., Suo, J., Wetzstein, G., Dai, Q., and Raskar, R. 2013. Coded focal stack photography. In Proc. IEEE ICCP, 1--9.Google ScholarGoogle Scholar
  28. Lin, X., Wetzstein, G., Liu, Y., and Dai, Q. 2014. Dual-coded compressive hyperspectral imaging. Optics Letters 39, 7, 2044--2047.Google ScholarGoogle ScholarCross RefCross Ref
  29. Ma, C., Cao, X., Tong, X., Dai, Q., and Lin, S. 2013. Acquisition of high spatial and spectral resolution video with a hybrid camera system. IJCV, 1--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Manakov, A., Restrepo, J. F., Klehm, O., Hegedüs, R., Eisemann, E., Seidel, H.-P., and Ihrke, I. 2013. A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging. ACM Trans. Graph. (SIGGRAPH) 32, 4, 47. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Marwah, K., Wetzstein, G., Bando, Y., and Raskar, R. 2013. Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Trans. Graph. (SIGGRAPH) 32, 4, 46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Mohan, A., Raskar, R., and Tumblin, J. 2008. Agile spectrum imaging: Programmable wavelength modulation for cameras and projectors. Computer Graphics Forum 27, 2, 709--717.Google ScholarGoogle ScholarCross RefCross Ref
  33. Natarajan, B. K. 1995. Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 2, 227--234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Pan, Z., Healey, G., Prasad, M., and Tromberg, B. 2003. Face recognition in hyperspectral images. IEEE Trans. PAMI 25, 12, 1552--1560. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Park, J.-I., Lee, M.-H., Grossberg, M. D., and Nayar, S. K. 2007. Multispectral imaging using multiplexed illumination. In Proc. IEEE ICCV, 1--8.Google ScholarGoogle Scholar
  36. Parmar, M., Lansel, S., and Wandell, B. A. 2008. Spatiospectral reconstruction of the multispectral datacube using sparse recovery. In Proc. IEEE ICIP, 473--476.Google ScholarGoogle Scholar
  37. Pham, T. H., Bevilacqua, F., Spott, T., Dam, J. S., Tromberg, B. J., and Andersson-Engels, S. 2000. Quantifying the absorption and reduced scattering coefficients of tissuelike turbid media over a broad spectral range with non-contact Fourier-transform hyperspectral imaging. Applied Optics 39, 34, 6487--6497.Google ScholarGoogle ScholarCross RefCross Ref
  38. Porter, W. M., and Enmark, H. T. 1987. A system overview of the airborne visible/infrared imaging spectrometer (AVIRIS). In Proc. SPIE, vol. 834, 22--31.Google ScholarGoogle Scholar
  39. Rajwade, A., Kittle, D., Tsai, T.-H., Brady, D., and Carin, L. 2013. Coded hyperspectral imaging and blind compressive sensing. SIAM J. Imaging Sci. 6, 2, 782--812.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Schechner, Y. Y., and Nayar, S. K. 2002. Generalized mosaicing: Wide field of view multispectral imaging. IEEE Trans. PAMI 24, 10, 1334--1348. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Smith, W. L., Zhou, D. K., Harrison, F. W., Revercomb, H. E., Larar, A. M., Huang, H.-L., and Huang, B. 2001. Hyperspectral remote sensing of atmospheric profiles from satellites and aircraft. In Proc. SPIE, 94--102.Google ScholarGoogle Scholar
  42. Van Den Berg, E., and Friedlander, M. P. 2008. Probing the pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31, 2, 890--912. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Wagadarikar, A. A., Pitsianis, N. P., Sun, X., and Brady, D. J. 2009. Video rate spectral imaging using a coded aperture snapshot spectral imager. Optics Express 17, 8, 6368--6388.Google ScholarGoogle ScholarCross RefCross Ref
  44. Wu, Y., Mirza, I. O., Arce, G. R., and Prather, D. W. 2011. Development of a digital-micromirror-device-based multishot snapshot spectral imaging system. Optics Letters 36, 14, 2692--2694.Google ScholarGoogle ScholarCross RefCross Ref
  45. Yamaguchi, M., Haneishi, H., Fukuda, H., Kishimoto, J., Kanazawa, H., Tsuchida, M., Iwama, R., and Ohyama, N. 2006. High-fidelity video and still-image communication based on spectral information: Natural vision system and its applications. In Proc. SPIE, 60620G--60620G.Google ScholarGoogle Scholar
  46. Zhou, C., and Nayar, S. K. 2011. Computational cameras: convergence of optics and processing. IEEE Trans. Im. Proc. 20, 12, 3322--3340. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Spatial-spectral encoded compressive hyperspectral imaging

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 33, Issue 6
      November 2014
      704 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/2661229
      Issue’s Table of Contents

      Copyright © 2014 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 19 November 2014
      Published in tog Volume 33, Issue 6

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader