Gesture recognition using Bezier curves for visualization navigation from registered 3-D data

Abstract

This paper presents a gesture recognition system for visualization navigation. Scientists are interested in developing interactive settings for exploring large data sets in an intuitive environment. The input consists of registered 3-D data. A geometric method using Bezier curves is used for the trajectory analysis and classification of gestures. The hand gesture speed is incorporated into the algorithm to enable correct recognition from trajectories with variations in hand speed. The method is robust and reliable: correct hand identification rate is 99.9% (from 1641 frames), modes of hand movements are correct 95.6% of the time, recognition rate (given the right mode) is 97.9%. An application to gesture-controlled visualization of 3D bioinformatics data is also presented.

Introduction

Large and complex data sets are produced at a more rapid pace than tools and algorithms for their processing, analysis, and exploration. For example, the National Institute of Health's (NIH) Visible Human project generated data sets of a single 3-D volume that consists of 12 billion elements. Nearly a terabyte of satellite data is produced on a daily basis. Advanced physics simulation at Lawrence Livermore National Laboratory (LLNL) is generating large data sets, which are expected to increase to one terabyte every $\text{5}\text{min}$ by 2004.

Among the tools used to help explore and understand large data, visualization aids in gaining greater insight into important physical parameters (such as temperature, height, stress, velocity or pressure) and in finding anomalies. Such anomalies often are not obvious to scientists during the automatic localization but are easily picked out with visual data exploration. Correct representation can drastically reduce the time needed for the analysis. As data becomes huge, it requires more time for processing. Even the latest supercomputers may require days or even weeks for computations. This makes real-time visualization mission-critical, since interesting properties can show up which allow scientists to adjust parameters of computation and restart it, if needed. Visualization is integrated into the process and is no longer just the last step.

State-of-the-art visualization displays keep pace with the data requirements. For instance, one of LLNL's “power walls” (Fig. 1(a)) is a 15-projector system that displays approximately 19.7 million pixels on a 16- by 8-ft screen. Systems like this allow for detailed data analysis and team collaboration. However, applying even simple commands to the data (such as zoom, rotation and translation) requires a secondary, or “background”, communication process between scientists working with the data (see the two standing by the screen in Fig. 1(a)) and the “operator” responsible for executing selected commands (sitting, left). This reduces productivity of the team and affects quality of presentations.

Therefore, scientists are interested in developing new, interactive settings for exploring their data in a more intuitive environment. A gesture-recognition system can interpret commands and supply data manipulation parameters to visualization software (Fig. 1(b)) without having an “operator” involved in the process.

Since the system is being developed as a front end for gesture-controlled, large-scale visualization and virtual reality manipulation, certain requirements and complications are apparent. First, 3-D information is required, not necessarily at a video-frame rate, but at least a few times per second (optimal parameters should be determined as a result of testing on a large group of people). Second, traditional techniques such as background subtraction cannot easily separate a figure from the background, since the entire body of the interacting person (not only arms or hands) is moving. Moreover, interaction takes place in front of the screen where the data is updated dynamically and, therefore, the background changes most of the time. Third, motion of the interacting person should be natural and should result in intuitive data manipulation, where intuitive means easily learned and fast to provide immediate results.

Gesture tracking and recognition are important research domains. Traditional approaches to tracking typically relied on segmentation of the intensity data, using motion or appearance data. A majority of the methods began by segmenting the human body from the background. For example, in “blob approaches”, people were modeled as a number of blobs resulting from pixel classification based on their color and position in the image. Wren et al. [1] achieved segmentation by classifying pixels into one of several models, including a static world and a dynamic user represented by Gaussian blobs. Yang and Ahuja [2] used skin color and the geometry of palm and face regions for segmentation stages of their system. A Gaussian mixture (with parameters estimated by an EM algorithm) modeled the distribution of skin-color pixels. Rehg and Kanade [3] used a 3-D hand model to track a hand. They compared line features from the images with the projected model and performed incremental state corrections. Similar work was presented by Kuch and Huang [4], in which the synthesis process could fit the hand model to any person's hand. Cutler and Davis [5] segmented the motion and computed a moving object's self-similarity (including human motion experiments).

A significant amount of work is being performed in the area of recognition, where Hidden Markov models (HMMs) are often employed successfully [6], [7], [8] by allowing researchers to address the highly stochastic nature of human gestures. Yacoob and Black proposed parameterized representation of human movement in the form of principal components [9]. Bobick and Wilson [10] treated gesture as a sequence of states and computed configuration states along prototype gestures. Yang and Ahuja [2] used motion trajectories for recognition. Grzeszcuk et al. [11] described classification algorithm with statistical moments of the binarized gesture templates. Hong et al. [12] treated each gesture as a finite state machine (FSM) in the spatial-temporal space; FSMs were trained using k-means clustering. A preliminary trained neural network was used by Sato et al. [13]. Hongo et al. [14] performed recognition by a linear discriminant analysis in each discriminant space by using four directional features. Approach described by Yoon et al. [15] derived features from location, angle, and velocity and employed a k-means clustering algorithm for the HMMs. Gesture contour representation and alignment-based classification were proposed by Gupta and Ma [16]. A review by Aggarwal and Cai [17] classified approaches to human motion analysis, the tasks involved, and major areas related to human motion interpretation. A review by Pavlovic et al. [18] addressed main components and directions in gesture recognition research for human-computer interaction (HCI).