Skip to main content
Top

2010 | Book

Animation and Performance Capture Using Digitized Models

insite
SEARCH

About this book

The realistic generation of virtual doubles of real-world actors has been the focus of computer graphics research for many years. However, some problems still remain unsolved: it is still time-consuming to generate character animations using the traditional skeleton-based pipeline, passive performance capture of human actors wearing arbitrary everyday apparel is still challenging, and until now, there is only a limited amount of techniques for processing and modifying mesh animations, in contrast to the huge amount of skeleton-based techniques. In this thesis, we propose algorithmic solutions to each of these problems. First, two efficient mesh-based alternatives to simplify the overall character animation process are proposed. Although abandoning the concept of a kinematic skeleton, both techniques can be directly integrated in the traditional pipeline, generating animations with realistic body deformations. Thereafter, three passive performance capture methods are presented which employ a deformable model as underlying scene representation. The techniques are able to jointly reconstruct spatio-temporally coherent time-varying geometry, motion, and textural surface appearance of subjects wearing loose and everyday apparel. Moreover, the acquired high-quality reconstructions enable us to render realistic 3D Videos. At the end, two novel algorithms for processing mesh animations are described. The first one enables the fully-automatic conversion of a mesh animation into a skeletonbased animation and the second one automatically converts a mesh animation into an animation collage, a new artistic style for rendering animations. The methods described in the thesis can be regarded as solutions to specific problems or important building blocks for a larger application. As a whole, they form a powerful system to accurately capture, manipulate and realistically render realworld human performances, exceeding the capabilities of many related capture techniques. By this means, we are able to correctly capture the motion, the timevarying details and the texture information of a real human performing, and transform it into a fully-rigged character animation, that can be directly used by an animator, or use it to realistically display the actor from arbitrary viewpoints.

Table of Contents

Frontmatter

Introduction

Introduction
Abstract
This book presents methods for the realistic generation of virtual doubles of real-world actors. First, two efficient mesh-based alternatives to simplify the overall character animation process are proposed. Thereafter, three passive performance capture methods are presented dealing with actors wearing arbitrary everyday apparel. At the end, two novel algorithms for processing mesh animations are described. As a whole, the methods presented here form a powerful system to accurately capture, manipulate and realistically render real-world human performances, exceeding the capabilities of many related capture techniques.
Edilson de Aguiar

Part I: Background and Basic Definitions

Frontmatter
Preliminary Techniques
Abstract
In this chapter, first some general theoretical background information about camera and human models are given. Thereafter, elementary computer vision techniques that many of the projects in this book capitalize on are described.
Edilson de Aguiar
Interactive Shape Deformation and Editing Methods
Abstract
This chapter reviews the most relevant work on interactive shape deformation and editing techniques, and describes the three main deformation approaches used in this book.
Edilson de Aguiar
Recording Studio: Data Acquisition and Data Processing
Abstract
This chapter describes our recording studio. First, the physical studio, the camera system, and the full body laser scanner are presented. Thereafter, the acquisition pipeline is detailed, with all necessary steps to generate the input data for the projects described in this book.
Edilson de Aguiar

Part II: Natural Animation of Digitized Models

Frontmatter
Problem Statement and Preliminaries
Abstract
This part reviews the steps involved in the traditional skeleton-based character animation paradigm, and proposes two mesh-based alternatives that simplify the conventional process.
Edilson de Aguiar
Poisson-Based Skeleton-Less Character Animation
Abstract
This chapter presents an approach to efficiently generate high-quality animations of human characters from input motion data. Using a Poisson-based deformation method, as described in Sect. 3.2.1, the proposed approach outputs character animations with realistic body deformations, only requiring a minimum of manual interaction.
Edilson de Aguiar
Laplacian-Based Skeleton-Less Character Animation
Abstract
This chapter extends the original mesh-based animation framework described in the previous chapter. By using a Laplacian-based scheme to guide the mesh deformation process, the improved system described here allows for more accurate control producing animations from motion capture data.
Edilson de Aguiar

Part III: Towards Performance Capture Using Deformable Mesh Tracking

Frontmatter
Problem Statement and Preliminaries
Abstract
This part of the book first reviews the most relevant work on motion capture, scene reconstruction, and 3D Video. Thereafter, it describes three algorithms to passively capture the performance of human actors, and presents a system to generate high-quality 3D Videos.
Edilson de Aguiar
Video-Based Tracking of Scanned Humans
Abstract
In this chapter, we propose our first performance capture algorithm. By combining an image-based 3D correspondence estimation algorithm and a Guided Laplacian-based mesh deformation scheme (Sect. 3.2.2), our system captures the performance of a moving subject from multiple video streams, while preserving the connectivity of the underlying mesh structure over time.
Edilson de Aguiar
Feature Tracking for Mesh-Based Performance Capture
Abstract
In this chapter, we propose our second performance capture framework. First, a robust method to track 3D trajectories of features on a moving subject recorded with multiple cameras is described. Thereafter, by combining the 3D trajectories with a mesh deformation scheme, the performance of a moving actor is captured and the high-quality scanned model can be directly animated such that it mimics the subject’s motion.
Edilson de Aguiar
Video-Based Performance Capture
Abstract
In this chapter, by combining the power of surface- and volume-based shape deformation techniques with a novel mesh-based analysis-through-synthesis framework, a new marker-less approach to capture human performances from multi-view video is described. Furthermore, a silhouette rim-based technique and a model-guided multi-view stereo approach are also presented, which enables the acquisition of fine dynamic subtle details more accurately.
Edilson de Aguiar
High-Quality 3D Videos
Abstract
In this chapter, we describe a method to render realistic 3D Videos by applying a clever dynamic 3D texturing scheme to the moving geometry representation captured by the methods proposed in the previous chapters. By displaying high-quality renderings of the recorded actor from any viewpoint, our system enables new interesting applications for 3D Television.
Edilson de Aguiar

Part IV: Processing Mesh Animations

Frontmatter
Problem Statement and Preliminaries
Abstract
In this part of the book, we propose two novel techniques to simplify the process of mesh animations. First, an automatic method to extract a plausible kinematic skeleton, skeletal motion parameters, and surface skinning weights from arbitrary mesh animations is presented. Thereafter, a method to automatically transform mesh animations into animation collages, i.e. moving assemblies of shape primitives from a database, is described.
Edilson de Aguiar
Reconstructing Fully-Rigged Characters
Abstract
In this chapter, we propose a method to automatically extract a plausible kinematic skeleton, skeletal motion parameters, and surface skinning weights from arbitrary mesh animations, bridging the gap between the mesh-based and the skeletal paradigms. By this means, sequences captured in the previous chapters can be automatically transformed into fully-rigged virtual characters.
Edilson de Aguiar
Designing Non-photorealistic Animation Collages
Abstract
In this chapter, we present a method to automatically transform mesh animations into animation collages, i.e. a new non-photorealistic rendering style for animations. By automatically decomposing input animations and fitting a shape from the database into each segment, our algorithm creates a new rendering style. It has many applications in arts, non-photorealistic rendering, and animated movie productions.
Edilson de Aguiar
Conclusions
Abstract
This book has presented novel algorithms to accurately capture, manipulate and realistically render real-world human performances, going beyond the limits of related capture techniques. Each method described in this work can be regarded as a specific solution to a challenging problem or as a building block that enables the development of novel applications. The methods outlined in this book have been originally tailored to deal with human actors. However, the fundamental principles are also applicable to a larger class of real-world scenes.
Edilson de Aguiar
Backmatter
Metadata
Title
Animation and Performance Capture Using Digitized Models
Author
Edilson de Aguiar
Copyright Year
2010
Publisher
Springer Berlin Heidelberg
Electronic ISBN
978-3-642-10316-2
Print ISBN
978-3-642-10315-5
DOI
https://doi.org/10.1007/978-3-642-10316-2