Skip to main content

Über dieses Buch

With the swift development of video imaging technology and the drastic improvements in CPU speed and memory, both video processing and computational video are becoming more and more popular. Similar to the digital revolution in photography of fifteen years ago, today digital methods are revolutionizing the way television and movies are being made. With the advent of professional digital movie cameras, digital projector technology for movie theaters, and 3D movies, the movie and
television production pipeline is turning all-digital, opening up
numerous new opportunities for the way dynamic scenes are acquired, video footage can be edited, and visual media may be experienced.

This state-of-the-art survey provides a compilation of selected articles resulting from a workshop on Video Processing and Computational Video, held at Dagstuhl Castle, Germany, in October 2010. The seminar brought together junior and senior
researchers from computer vision, computer graphics, and image communication, both from academia and industry, to address the challenges in computational video. During this workshop, 43 researchers from all over the world discussed the state of the art, contemporary challenges, and future research in imaging,
processing, analyzing, modeling, and rendering of real-world, dynamic scenes.

The 8 thoroughly revised papers presented were carefully reviewed and selected from more than 30 lectures given at the seminar. The articles give a good overview of the field of computational video and video processing with a special
focus on computational photography, video-based rendering, and 3D video.



Video Processing and Computational Video

Towards Plenoptic Raumzeit Reconstruction

The goal of image-based rendering is to evoke a visceral sense of presense in a scene using only photographs or videos. A huge variety of different approaches have been developed during the last decade. Examining the underlying models we find three different main categories: view interpolation based on geometry proxies, pure image interpolation techniques and complete scene flow reconstruction. In this paper we present three approaches for free-viewpoint video, one for each of these categories and discuss their individual benefits and drawbacks. We hope that studying the different approaches will help others in making important design decisions when planning a free-viewpoint video system.
Martin Eisemann, Felix Klose, Marcus Magnor

Two Algorithms for Motion Estimation from Alternate Exposure Images

Most algorithms for dense 2D motion estimation assume pairs of images that are acquired with an idealized, infinitively short exposure time. In this work we compare two approaches that use an additional, motion-blurred image of a scene to estimate highly accurate, dense correspondence fields.
We consider video sequences that are acquired with alternating exposure times so that a short-exposure image is followed by a long-exposure image that exhibits motion-blur. For both motion estimation algorithms we employ an image formation model that relates the motion blurred image to two enframing short-exposure images. With this model we can decipher the motion information encoded in the long-exposure image, but also estimate occlusion timings which are a prerequisite for artifact-free frame interpolation. The first approach solves for the motion in a pointwise least squares formulation while the second formulates a global, total variation regularized problem. Both approaches are evaluated in detail and compared to each other and state-of-the-art motion estimation algorithms.
Anita Sellent, Martin Eisemann, Marcus Magnor

Understanding What we Cannot See: Automatic Analysis of 4D Digital In-Line Holographic Microscopy Data

Digital in-line holography is a microscopy technique which got an increasing attention over the last few years in the fields of microbiology, medicine and physics, as it provides an efficient way of measuring 3D microscopic data over time. In this paper, we present a complete system for the automatic analysis of digital in-line holographic data; we detect the 3D positions of the microorganisms, compute their trajectories over time and finally classify these trajectories according to their motion patterns. Tracking is performed using a robust method which evolves from the Hungarian bipartite weighted graph matching algorithm and allows us to deal with newly entering and leaving particles and compensate for missing data and outliers. In order to fully understand the behavior of the microorganisms, we make use of Hidden Markov Models (HMMs) to classify four different motion patterns of a microorganism and to separate multiple patterns occurring within a trajectory. We present a complete set of experiments which show that our tracking method has an accuracy between 76% and 91%, compared to ground truth data. The obtained classification rates on four full sequences (2500 frames) range between 83.5% and 100%.
Laura Leal-Taixé, Matthias Heydt, Axel Rosenhahn, Bodo Rosenhahn

3D Reconstruction and Video-Based Rendering of Casually Captured Videos

In this chapter we explore the possibility of interactively navigating a collection of casually captured videos of a performance: real-world footage captured on hand held cameras by a few members of the audience. The aim is to navigate the video collection in 3D by generating video based rendering of the performance using the offline pre-computed reconstruction of the event.
We propose two different techniques to obtain this reconstruction, considering that the video collection may have been recorded in complex, uncontrolled outdoor environments. One approach recovers the event geometry by exploring the temporal domain of each video independently, while the other explores the spatial domain of the video collection at each time instant, independently. The pros and cons of the two methods and their applicability to the addressed navigation problem, are also discussed. In the end, we propose an interactive GPU-accelerated viewing tool to navigate the video collection.
Aparna Taneja, Luca Ballan, Jens Puwein, Gabriel J. Brostow, Marc Pollefeys

Silhouette-Based Variational Methods for Single View Reconstruction

We explore the 3D reconstruction of objects from a single view within an interactive framework by using silhouette information. In order to deal with the highly ill-posed nature of the problem we propose two different reconstruction priors: a shape and a volume prior and cast them into a variational problem formulation. For both priors we show that the corresponding relaxed optimization problem is convex. This leads to unique solutions which are independent of initialization and which are either globally optimal (shape prior) or can be shown to lie within bounds from the optimal solution (volume prior). We analyze properties of the proposed priors with regard to the reconstruction results as well as their impact on the minimization problem. By employing an implicit volumetric representation our reconstructions enjoy complete topological freedom. Being parameter-based, our interactive reconstruction tool allows for intuitive and easy to use modeling of the reconstruction result.
Eno Töppe, Martin R. Oswald, Daniel Cremers, Carsten Rother

Single Image Blind Deconvolution with Higher-Order Texture Statistics

We present a novel method for solving blind deconvolution, i.e., the task of recovering a sharp image given a blurry one. We focus on blurry images obtained from a coded aperture camera, where both the camera and the scene are static, and allow blur to vary across the image domain. As most methods for blind deconvolution, we solve the problem in two steps: First, we estimate the coded blur scale at each pixel; second, we deconvolve the blurry image given the estimated blur. Our approach is to use linear high-order priors for texture and second-order priors for the blur scale map, i.e., constraints involving two pixels at a time. We show that by incorporating the texture priors in a least-squares energy minimization we can transform the initial blind deconvolution task in a simpler optimization problem. One of the striking features of the simplified optimization problem is that the parameters that define the functional can be learned offline directly from natural images via singular value decomposition. We also show a geometrical interpretation of image blurring and explain our method from this viewpoint. In doing so we devise a novel technique to design optimally coded apertures. Finally, our coded blur identification results in computing convolutions, rather than deconvolutions, which are stable operations. We will demonstrate in several experiments that this additional stability allows the method to deal with large blur. We also compare our method to existing algorithms in the literature and show that we achieve state-of-the-art performance with both synthetic and real data.
Manuel Martinello, Paolo Favaro

Compressive Rendering of Multidimensional Scenes

Recently, we proposed the idea of using compressed sensing to reconstruct the 2D images produced by a rendering system, a process we called compressive rendering. In this work, we present the natural extension of this idea to multidimensional scene signals as evaluated by a Monte Carlo rendering system. Basically, we think of a distributed ray tracing system as taking point samples of a multidimensional scene function that is sparse in a transform domain. We measure a relatively small set of point samples and then use compressed sensing algorithms to reconstruct the original multidimensional signal by looking for sparsity in a transform domain. Once we reconstruct an approximation to the original scene signal, we can integrate it down to a final 2D image which is output by the rendering system. This general form of compressive rendering allows us to produce effects such as depth-of-field, motion blur, and area light sources, and also renders animated sequences efficiently.
Pradeep Sen, Soheil Darabi, Lei Xiao

Efficient Rendering of Light Field Images

Recently a new display type has emerged that is able to display 50,000 views offering a full parallax autostereoscopic view of static scenes. With the advancement in the manufacturing technology, multi-view displays come with more and more views of dynamic content, closing the gap to this high quality full parallax display.
The established method of content generation for synthetic stereo images is to render both views. To ensure a high quality these images are often ray traced. With the increasing number of views, rendering of all views is not feasible for multi-view displays.
Therefore methods are required that can render the large amount of different views required by those displays efficiently. In the following a complete solution is presented that describes how all views for a full parallax display can be rendered from a small set of input images and their associated depth images with an image-based rendering algorithm.
An acceleration of the rendering of two orders of magnitude is achieved by different parallelization techniques and the use of efficient data structures.
Moreover, the problem of finding the best-next-view for an image-based rendering algorithm is addressed and a solution is presented that ranks possible viewpoints based on their suitability for an image-based rendering algorithm.
Daniel Jung, Reinhard Koch


Weitere Informationen

Premium Partner