Handbook of Convex Optimization Methods in Imaging Science

Images and videos are ubiquitous in our multimedia-rich environment today. We can capture photographs on a variety of devices, from inexpensive mobile phones to highly sophisticated cameras, spanning an impressive range in sensor pixel-count and picture resolution. We watch video content on displays ranging in size from handheld devices to gigantic movie screens. The resolution on these videos ranges from the grainy CIF (352 × 288) in CCTVs all the way up to 8K ultra high-definition (7680 × 4320) and beyond. If that is not impressive enough, devices routinely support high-quality streaming video content, while preserving the richness of spectral or color information. By some estimates, the global consumer electronics market is poised to be worth a mind-boggling trillion US dollars by 2020.

Color quality and fidelity are fundamental considerations in today’s digital imaging systems. Optimization of a color imaging system is a multifaceted problem involving deep understanding of device physics, light-surface interactions, human visual perception, and computational mathematics. The design of a successful color imaging system that meets the desired performance, reliability and cost evokes many interesting and challenging optimization problems. This chapter explores a variety of optimization frameworks that have been developed for color capture, display, and printing. For each device genre, a broad introduction to challenges in color imaging is first presented, followed by a detailed exposition of selected optimization problems and their solutions. Practical considerations such as computational cost, noise containment, and power consumption are introduced as mathematical constraints into the given optimization problem. The chapter concludes with suggestions for future work in this domain.

We review recent developments in Synthetic Aperture Radar (SAR) image formation from an optimization perspective. Majority of these methods can be viewed as constrained least squares problems exploiting sparsity. We reviewed analytic and large scale numerical optimization based approaches in both deterministic and Bayesian frameworks. These methods offer substantial improvements in image quality, suppression of noise and clutter. Analytic methods also have the advantage of computational efficiency.

Multidimensional optical imaging, that is, capturing light in more than two-dimensions (unlike conventional photography), has been an emerging field with widespread applications in diverse domains. Due to the intrinsic limitation of two-dimensional detectors in capturing inherently higher-dimensional data, multidimensional imaging techniques conventionally rely on a scanning process, which renders them inefficient in terms of light throughput and unsuitable for dynamic scenes. In this chapter, we present recent multidimensional imaging techniques for spectral and temporal imaging, which overcome the temporal, spectral, and spatial resolution limitations of conventional scanning-based systems. Each development is based on the computational imaging paradigm, which involves distributing the imaging task between a physical and a computational system and then digitally forming the image datacube of interest from multiplexed measurements by means of solving an inverse problem via convex optimization techniques.

In recent years, Sparse Representation (SR) and Dictionary Learning (DL) have emerged as powerful tools for efficient processing image and video data in non-traditional ways. An area of promise for these theories is object recognition. In this chapter, we review the role of algorithms based on SR and DL for object recognition. In particular, supervised, unsupervised, weakly supervised, nonlinear kernel-based, convolutional sparse coding and analysis DL methods are reviewed.

There are many rich connections between the theory of mathematical optimization and the practice of image restoration. However, several fundamental questions remain open—e.g., how to translate some physical insight into an appropriate mathematical objective/cost functional? what kind of optimization tools should be called on first? The objective of this chapter is to stress the difference between the theory and the practice—namely, in the practice of image restoration, the objective is often not to solve the formulated optimization problem correctly but to obtain a nicely-restored image through the process of optimization. In other words, we advocate the termination of an iterative optimization algorithm before it reaches the convergence for various practical considerations (e.g., computational constraints, regularization purpose). Meanwhile, we will show that strategies such as relaxation and divide-and-conquer—even though they do not help the pursuit of a globally optimal solution—are often sufficient for the applications of image restoration. We will use two specific applications—namely image denoising and compressed sensing—to demonstrate how simultaneous sparse coding and nonlocal regularization both admit a nonconvex optimization-based formulation, which can lead to novel insights to our understanding why BM3D and BM3D-CS can achieve excellent performance.

Over the past decade, sparsity has emerged as a dominant theme in signal processing and big data applications. In this chapter, we formulate and solve new flavors of sparsity-constrained optimization problems built on the family of spike-and-slab priors. First, we develop an efficient Iterative Convex Refinement solution to the hard non-convex problem of Bayesian signal recovery under sparsity-inducing spike-and-slab priors. We also offer a Bayesian perspective on sparse representation-based classification via the introduction of class-specific priors. This formulation represents a consummation of ideas developed for model-based compressive sensing into a general framework for sparse model-based classification.