Abstract
This paper proposes a novel compressive hyperspectral (HS) imaging approach that allows for high-resolution HS images to be captured in a single image. The proposed architecture comprises three key components: spatial-spectral encoded optical camera design, over-complete HS dictionary learning and sparse-constraint computational reconstruction. Our spatial-spectral encoded sampling scheme provides a higher degree of randomness in the measured projections than previous compressive HS imaging approaches; and a robust nonlinear sparse reconstruction method is employed to recover the HS images from the coded projection with higher performance. To exploit the sparsity constraint on the nature HS images for computational reconstruction, an over-complete HS dictionary is learned to represent the HS images in a sparser way than previous representations. We validate the proposed approach on both synthetic and real captured data, and show successful recovery of HS images for both indoor and outdoor scenes. In addition, we demonstrate other applications for the over-complete HS dictionary and sparse coding techniques, including 3D HS images compression and denoising.
Supplemental Material
Available for Download
Supplemental material.
- Aharon, M., Elad, M., and Bruckstein, A. 2006. K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Proc. 54, 11, 4311--4322. Google ScholarDigital Library
- Arguello, H., Rueda, H., Wu, Y., Prather, D. W., and Arce, G. R. 2013. Higher-order computational model for coded aperture spectral imaging. Applied Optics 52, 10, D12--D21.Google ScholarCross Ref
- August, Y., Vachman, C., Rivenson, Y., and Stern, A. 2013. Compressive hyperspectral imaging by random separable projections in both the spatial and the spectral domains. Applied Optics 52, 10, D46--D54.Google ScholarCross Ref
- Basedow, R. W., Carmer, D. C., and Anderson, M. E. 1995. Hydice system: Implementation and performance. In Proc. SPIE, 258--267.Google Scholar
- Candes, E. J., and Tao, T. 2005. Decoding by linear programming. IEEE Trans. Inform. Theory 51, 12, 4203--4215. Google ScholarDigital Library
- Candes, E. J., Eldar, Y. C., Needell, D., and Randall, P. 2011. Compressed sensing with coherent and redundant dictionaries. Appl. Comput. Harmon. Anal. 31, 1, 59--73.Google ScholarCross Ref
- Chakrabarti, A., and Zickler, T. 2011. Statistics of real-world hyperspectral images. In Proc. IEEE CVPR, 193--200. Google ScholarDigital Library
- Chi, C., Yoo, H., and Ben-Ezra, M. 2010. Multi-spectral imaging by optimized wide band illumination. IJCV 86, 2-3, 140--151. Google ScholarDigital Library
- Donoho, D. L., and Huo, X. 2001. Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inform. Theory 47, 7, 2845--2862. Google ScholarDigital Library
- Donoho, D. L. 2006. Compressed sensing. IEEE Trans. Inform. Theory 52, 4, 1289--1306. Google ScholarDigital Library
- Du, H., Tong, X., Cao, X., and Lin, S. 2009. A prism-based system for multispectral video acquisition. In Proc. IEEE ICCV, 175--182.Google Scholar
- Duarte-Carvajalino, J. M., and Sapiro, G. 2009. Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization. IEEE Trans. Im. Proc. 18, 7, 1395--1408. Google ScholarDigital Library
- Elad, M., and Aharon, M. 2006. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Im. Proc. 15, 12, 3736--3745. Google ScholarDigital Library
- Elad, M. 2007. Optimized projections for compressed sensing. IEEE Trans. Signal Proc. 55, 12, 5695--5702. Google ScholarDigital Library
- Gao, L., Kester, R. T., Hagen, N., and Tkaczyk, T. S. 2010. Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy. Optics Express 18, 14, 14330--14344.Google ScholarCross Ref
- Gat, N., Scriven, G., Garman, J., De Li, M., and Zhang, J. 2006. Development of four-dimensional imaging spectrometers (4D-IS). In Proc. SPIE Optics+ Photonics, 63020M--63020M.Google Scholar
- Gat, N. 2000. Imaging spectroscopy using tunable filters: a review. In Proc. AeroSense, 50--64.Google Scholar
- Gehm, M., John, R., Brady, D., Willett, R., and Schulz, T. 2007. Single-shot compressive spectral imaging with a dual-disperser architecture. Optics Express 15, 21, 14013--14027.Google ScholarCross Ref
- Gorman, A., Fletcher-Holmes, D. W., and Harvey, A. R. 2010. Generalization of the lyot filter and its application to snapshot spectral imaging. Optics Express 18, 6, 5602--5608.Google ScholarCross Ref
- Han, S., Sato, I., Okabe, T., and Sato, Y. 2011. Fast spectral reflectance recovery using DLP projector. In Proc. ACCV, 323--335. Google ScholarDigital Library
- Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., and Nayar, S. K. 2011. Video from a single coded exposure photograph using a learned over-complete dictionary. In Proc. IEEE ICCV, 287--294. Google ScholarDigital Library
- Hullin, M. B., Hanika, J., Ajdin, B., Seidel, H.-P., Kautz, J., and Lensch, H. 2010. Acquisition and analysis of bispectral bidirectional reflectance and reradiation distribution functions. ACM Trans. Graph. (SIGGRAPH) 29, 4, 97. Google ScholarDigital Library
- Johnson, W. R., Wilson, D. W., and Bearman, G. 2006. Spatial-spectral modulating snapshot hyperspectral imager. Applied Optics 45, 9, 1898--1908.Google ScholarCross Ref
- Kawakami, R., Wright, J., Tai, Y.-W., Matsushita, Y., Ben-Ezra, M., and Ikeuchi, K. 2011. High-resolution hyperspectral imaging via matrix factorization. In Proc. IEEE CVPR, 2329--2336. Google ScholarDigital Library
- Kim, M. H., Harvey, T. A., Kittle, D. S., Rushmeier, H., Dorsey, J., Prum, R. O., and Brady, D. J. 2012. 3D imaging spectroscopy for measuring hyperspectral patterns on solid objects. ACM Trans. Graph. (SIGGRAPH) 31, 4, 38. Google ScholarDigital Library
- Kittle, D., Choi, K., Wagadarikar, A., and Brady, D. J. 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Applied Optics 49, 36, 6824--6833.Google ScholarCross Ref
- Lin, X., Suo, J., Wetzstein, G., Dai, Q., and Raskar, R. 2013. Coded focal stack photography. In Proc. IEEE ICCP, 1--9.Google Scholar
- Lin, X., Wetzstein, G., Liu, Y., and Dai, Q. 2014. Dual-coded compressive hyperspectral imaging. Optics Letters 39, 7, 2044--2047.Google ScholarCross Ref
- Ma, C., Cao, X., Tong, X., Dai, Q., and Lin, S. 2013. Acquisition of high spatial and spectral resolution video with a hybrid camera system. IJCV, 1--15. Google ScholarDigital Library
- Manakov, A., Restrepo, J. F., Klehm, O., Hegedüs, R., Eisemann, E., Seidel, H.-P., and Ihrke, I. 2013. A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging. ACM Trans. Graph. (SIGGRAPH) 32, 4, 47. Google ScholarDigital Library
- Marwah, K., Wetzstein, G., Bando, Y., and Raskar, R. 2013. Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Trans. Graph. (SIGGRAPH) 32, 4, 46. Google ScholarDigital Library
- Mohan, A., Raskar, R., and Tumblin, J. 2008. Agile spectrum imaging: Programmable wavelength modulation for cameras and projectors. Computer Graphics Forum 27, 2, 709--717.Google ScholarCross Ref
- Natarajan, B. K. 1995. Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 2, 227--234. Google ScholarDigital Library
- Pan, Z., Healey, G., Prasad, M., and Tromberg, B. 2003. Face recognition in hyperspectral images. IEEE Trans. PAMI 25, 12, 1552--1560. Google ScholarDigital Library
- Park, J.-I., Lee, M.-H., Grossberg, M. D., and Nayar, S. K. 2007. Multispectral imaging using multiplexed illumination. In Proc. IEEE ICCV, 1--8.Google Scholar
- Parmar, M., Lansel, S., and Wandell, B. A. 2008. Spatiospectral reconstruction of the multispectral datacube using sparse recovery. In Proc. IEEE ICIP, 473--476.Google Scholar
- Pham, T. H., Bevilacqua, F., Spott, T., Dam, J. S., Tromberg, B. J., and Andersson-Engels, S. 2000. Quantifying the absorption and reduced scattering coefficients of tissuelike turbid media over a broad spectral range with non-contact Fourier-transform hyperspectral imaging. Applied Optics 39, 34, 6487--6497.Google ScholarCross Ref
- Porter, W. M., and Enmark, H. T. 1987. A system overview of the airborne visible/infrared imaging spectrometer (AVIRIS). In Proc. SPIE, vol. 834, 22--31.Google Scholar
- Rajwade, A., Kittle, D., Tsai, T.-H., Brady, D., and Carin, L. 2013. Coded hyperspectral imaging and blind compressive sensing. SIAM J. Imaging Sci. 6, 2, 782--812.Google ScholarDigital Library
- Schechner, Y. Y., and Nayar, S. K. 2002. Generalized mosaicing: Wide field of view multispectral imaging. IEEE Trans. PAMI 24, 10, 1334--1348. Google ScholarDigital Library
- Smith, W. L., Zhou, D. K., Harrison, F. W., Revercomb, H. E., Larar, A. M., Huang, H.-L., and Huang, B. 2001. Hyperspectral remote sensing of atmospheric profiles from satellites and aircraft. In Proc. SPIE, 94--102.Google Scholar
- Van Den Berg, E., and Friedlander, M. P. 2008. Probing the pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31, 2, 890--912. Google ScholarDigital Library
- Wagadarikar, A. A., Pitsianis, N. P., Sun, X., and Brady, D. J. 2009. Video rate spectral imaging using a coded aperture snapshot spectral imager. Optics Express 17, 8, 6368--6388.Google ScholarCross Ref
- Wu, Y., Mirza, I. O., Arce, G. R., and Prather, D. W. 2011. Development of a digital-micromirror-device-based multishot snapshot spectral imaging system. Optics Letters 36, 14, 2692--2694.Google ScholarCross Ref
- Yamaguchi, M., Haneishi, H., Fukuda, H., Kishimoto, J., Kanazawa, H., Tsuchida, M., Iwama, R., and Ohyama, N. 2006. High-fidelity video and still-image communication based on spectral information: Natural vision system and its applications. In Proc. SPIE, 60620G--60620G.Google Scholar
- Zhou, C., and Nayar, S. K. 2011. Computational cameras: convergence of optics and processing. IEEE Trans. Im. Proc. 20, 12, 3322--3340. Google ScholarDigital Library
Index Terms
- Spatial-spectral encoded compressive hyperspectral imaging
Recommendations
High-quality hyperspectral reconstruction using a spectral prior
We present a novel hyperspectral image reconstruction algorithm, which overcomes the long-standing tradeoff between spectral accuracy and spatial resolution in existing compressive imaging approaches. Our method consists of two steps: First, we learn ...
Compressive Hyperspectral Imaging Reconstruction by Spatial and Spectral Joint Prior
ICIIP '18: Proceedings of the 3rd International Conference on Intelligent Information ProcessingHyperspectral imaging systems can benefit from compressed sensing to reduce the size demand of sensor array. A new reconstruction algorithm is presented to recover the hyperspectral images from limited compressive measurements, exploiting the inherent ...
A novel joint dictionary framework for sparse hyperspectral unmixing incorporating spectral library
AbstractThe dictionary-aided sparse representation has recently become a promising method in hyperspectral unmixing. Under the assumption of linear mixture model, sparse unmixing aims at selecting a small subset of spectral signals in the ...
Comments