Advances in Visual Computing

Ontology-based software and image processing engine must cooperate in new fields of computer vision like microscopy acquisition wherein the amount of data, concepts and processing to be handled must be properly controlled. Within our own platform, we need to extract biological objects of interest in huge size and high-content microscopy images. In addition to specific low-level image analysis procedures, we used knowledge formalization tools and high-level reasoning ability of ontology-based software. This methodology made it possible to improve the expressiveness of the clinical models, the usability of the platform for the pathologist and the sensitivity or sensibility of the low-level image analysis algorithms.

Attribute-filtering, relying on the notion of component-tree, enables to process grey-level images by taking into account high-level

a priori

knowledge. Based on these notions, a method is proposed for automatic segmentation of vascular structures from phase-contrast magnetic resonance angiography. Experiments performed on 16 images and validations by comparison to results obtained by two human experts emphasise the relevance of the method.

We have developed a new semi-automated method for segmenting images of biological cells seeded at low density on tissue culture substrates, which we use to improve the generation of reference data for the evaluation of automated segmentation algorithms. The method was designed to mimic manual cell segmentation and is based on a model of human visual perception. We demonstrate a need for automated methods to assist with the generation of reference data by comparing several sets of masks from manually segmented cell images created by multiple independent hand-selections of pixels that belong to cell edges. We quantify the differences in these manually segmented masks and then compare them with masks generated from our new segmentation method which we use on cell images acquired to ensure very sharp, clear edges. The resulting masks from 16 images contain 71 cells and show that our semi-automated method for reference data generation locates cell edges more consistently than manual segmentation alone and produces better edge detection than other techniques like 5-means clustering and active contour segmentation for our images.

Image Registration is a central task to many medical image analysis applications. In this paper, we present a novel iterative algorithm composed of two main steps: a global affine image registration based on particle filter, and a local refinement obtained from a linear optical flow approximation. The key idea is to iteratively apply these simple and robust steps to efficiently solve complex non-rigid multimodal or unimodal image registrations. Finally, we present a set of evaluation experiments demonstrating the accuracy and applicability of the method to medical images.

In neurobiology, morphological change of neuronal structures such as dendrites and spines is important for understanding of brain functions or neuro-degenerative diseases. Especially, morphological changes of branching patterns of dendrites and volumetric spine structure is related to cognitive functions such as experienced-based learning, attention, and memory. To quantify their morphologies, we use confocal microscopy images which enables us to observe cellular structure with high resolution and three-dimensionally. However, the image resolution and field of view of microscopy is inversely proportional to the field of view (FOV) we cannot capture the whole structure of dendrite at on image. Therefore we combine partially obtained several images into a large image using image stitching techniques. To fine the overlapping region of adjacent images we use Fourier transform based phase correlation method. Then, we applied intensity blending algorithm to remove uneven intensity distribution and seam artifact at image boundaries which is coming from optical characteristics of microscopy. Finally, based on the integrated image we measure the morphology of dendrites from the center of cell to end of each branch. And geometrical characteristics of spine such as area, location, perimeter, and roundness, etc. are also quantified. Proposed method is fully automatic and provides accurate analysis of both local and global structural variations of neuron.

With a greater number of real-time graphics applications moving over to parametric surfaces from the polygonal domain, there is an inherent need to address various rendering bottlenecks that could hamper the move. Scaling the polygon count over various hardware platforms becomes an important factor. Much control is needed over the tessellation levels, either imposed by the hardware limitations or by the application. Developers like to create applications that run on various platforms without having to switch between polygonal and parametric versions to satisfy the limitations. In this paper, we present SD-2 (Semi-uniform, 2-Different), an adaptive tessellation algorithm for triangular parametric surfaces. The algorithm produces well distributed and semi-uniformly shaped triangles as a result of the tessellation. The SD-2 pattern requires new approaches for determining the edge tessellation factors, which can be fractional and change continuously depending on view parameters. The factors are then used to steer the tessellation of the parametric surface into a collection of triangle strips in a single pass. We compare the tessellation results in terms of GPU performance and surface quality by implementing SD-2 on PN patches.

Fast, reliable, and feature preserving automatic decimation of polygonal models is a challenging task. Exploiting both local volume and normal field variations in a novel way, a two phase decimation algorithm is proposed. In the first phase, a vertex is selected randomly using the measure of geometric fidelity that is based on normal field variation across its one-ring neighborhood. The selected vertex is eliminated in the second phase by collapsing an outgoing half-edge that is chosen by using volume based measure of geometric deviation. The proposed algorithm not only has better speed-quality trade-off but also keeps visually important features even after drastic simplification in a better way than similar state-of-the-art best algorithms; subjective and objective comparisons validate the assertion. This method can simplify huge models efficiently and is useful for applications where computing coordinates and/or attributes other than those attached to the original vertices is not allowed by the application and the focus is on both speed and quality of LODs.

A model for real-time generation of deep-water waves is suggested. It is based on a lattice-Boltzmann (LB) technique. Computation of wave dynamics and (ray-traced) rendering for a lattice of size 1024

2

can be carried out simultaneously on a single graphics card at 25 frames per second. In addition to the computational speed, the LB technique is seen to offer a simple and physically accurate method for handling both dispersion and wave reflection from obstructing objects.

We present a texture-based approach that hatches color photographs. We use a Delaunay triangulation to create a mesh of triangles with sizes that reflect the structure of an input image. At each vertex of this triangulation, the flow of the image is analyzed and a hatching texture is then created with the same alignment, based on real pencil strokes. This texture is given a modified version of a color sampled from the image, and then it is used to fill all the triangles adjoining the vertex. The three hatching textures that accumulate in each triangle are averaged, and the result of this process across all the triangles form the output image. This method can produce visually pleasing hatching similar to that seen in colored-pencil strokes and oil paintings.

A novel linear algorithm to estimate the camera pose from known correspondences of 3D points and their 2D image points is proposed based on the angle constraints from arbitrary three points in 3D point set. Compared with Ansar’s N Point Linear method which is based on the distance constraints between 3D points, due to more strict geometric constraints, this approach is more accurate. Simultaneously some strategies of choosing constraint equations are introduced so that this algorithm’s computational complexity is reduced. In order to obtain more accurate estimated pose, we propose the singular value decomposition method to derive the parameters from their quadratic terms more exactly. Finally, the experiments show our approach’s effectiveness and accuracy compared with the other two algorithms using synthetic data and real images.

We present a feature-preserving 3D thumbnail system for efficient 3D models database browsing. The 3D thumbnail is simplified from the original model so it requires much less hardware resource and transferring time. With topology-preserved 3D thumbnails, the user can browse multiple 3D models at once, and view each model from different angles interactively. To well preserve the topology of the original model, we propose an innovative voxel-based shape decomposition approach, which identifies meaningful parts of a 3D object, the 3D thumbnail is then created by approximating each individual part with fitting primitives. Experimental results demonstrates that the proposed approach can decompose a 3D model well to create a feature-preserving 3D thumbnail.

We present a novel framework to reliably learn scene entry and exit locations using coherent motion regions formed by weak tracking data. We construct “entities” from weak tracking data at a frame level and then track the entities through time, producing a set of consistent spatio-temporal paths. Resultant entity entry and exit observations of the paths are then clustered and a reliability metric is used to score the behavior of each entry and exit zone. We present experimental results from various scenes and compare against other approaches.

We investigate several methods of integrating facial actions into a 3D head model for 2D image search. The model on which the investigation is based has a neutral expression with eyes open, and our modifications enable the model to change expression and close the eyes. We show that the novel approach of using separate identity and action models during search gives better results than a combined-model strategy. This enables monitoring of head and feature movements in difficult real-world video sequences, which show large pose variation, occlusion, and variable lighting within and between frames. This should enable the identification of critical situations such as tiredness and inattention and we demonstrate the potential of our system by linking model parameters to states such as eyes closed and mouth open. We also present evidence that restricting the model parameters to a subspace close to the identity of the subject improves results.

Recent methods for human action recognition have been effective using increasingly complex, computationally-intensive models and algorithms. There has been growing interest in automated video analysis techniques which can be deployed onto resource-constrained distributed smart camera networks. In this paper, we introduce a multi-stage method for recognizing human actions (e.g., kicking, sitting, waving) that uses the motion patterns of easy-to-compute, low-level image features. Our method is designed for use on resource-constrained devices and can be optimized for real-time performance. In single-view and multi-view experiments, our method achieves 78% and 84% accuracy, respectively, on a publicly available data set.

Crowd behavior analysis is a challenging task for computer vision. In this paper, we present a novel approach for crowd behavior analysis and anomaly detection in coherent and incoherent crowded scenes. Two main aspects describe the novelty of the proposed approach: first, modeling the observed flow field in each non-overlapping block through social entropy to measure the concerning uncertainty of underlying field. Each block serves as an independent social system and social entropy determine the optimality criteria. The resulted in distributions of the flow field in respective blocks are accumulated statistically and the flow feature vectors are computed. Second, Support Vector Machines are used to train and classify the flow feature vectors as normal and abnormal. Experiments are conducted on two benchmark datasets PETS 2009 and University of Minnesota to characterize the specific and overall behaviors of crowded scenes. Our experiments show promising results with 95.6% recognition rate for both the normal and abnormal behavior in coherent and incoherent crowded scenes. Additionally, the similar method is tested using flow feature vectors without incorporating social entropy for comparative analysis and the detection results indicate the dominating performance of the proposed approach.

Camera based fall detection represents a solution to the problem of people falling down and being not able to stand up on their own again. For elderly people who live alone, such a fall is a major risk. In this paper we present an approach for fall detection based on multiple cameras supported by a statistical behavior model. The model describes the spatio-temporal unexpectedness of objects in a scene and is used to verify a fall detected by a semantic driven fall detection. In our work a fall is detected using multiple cameras where each of the camera inputs results in a separate fall confidence. These confidences are then combined into an overall decision and verified with the help of the statistical behavior model. This paper describes the fall detection approach as well as the verification step and shows results on 73 video sequences.

Multispectral datasets are becoming increasingly common. Consequently, effective techniques to deal with this kind of data are highly sought after. In this paper, we consider the problem of joint visualisation of multispectral datasets. Several improvements to existing methods are suggested leading to a new visualisation algorithm. The proposed algorithm also produces colour images, compared to grayscale images obtained through previous methods.

The color gamut of printing devices is generally smaller than that of imaging devices. Therefore, vivid color images cannot be reproduced on printed materials. This paper proposes a color gamut extension system for printed images by using a projector-camera system. The proposed system can capture a printed image using a video camera and extend the color gamut of the printed image by super-imposing a compensation image obtained from a projector device on the printed image. The compensation image is produced in the hue, saturation, and value (HSV) color space for iteratively adjusting the saturation and brightness values toward the boundary of the color gamut in the projector-camera system at each pixel. The feasibility of the proposed system is verified by experiments performed using real printed images.

This paper presents a new automatic method to significantly attenuate the color degradation due to shading in color images of the human skin. Shading is caused by illumination variation across the scene due to changes in local surface orientation, lighting conditions, and other factors. Our approach is to estimate the illumination variation by modeling it with a quadric function, and then relight the skin pixels with a simple operation. Therefore, the subsequent color skin image processing and analysis is simplified in several applications. We illustrate our approach in two typical color imaging problems involving human skin, namely: (a) pigmented skin lesion segmentation, and (b) face detection. Our preliminary experimental results show that our shading attenuation approach helps reducing the complexity of the color image analysis problem in these applications.

Brightness and color constancy is a fundamental problem faced in computer vision and by our own visual system. We easily recognize objects despite changes in illumination, but without a mechanism to cope with this, many object and face recognition systems perform poorly. In this paper we compare approaches in computer vision and computational neuroscience for inducing brightness and color constancy based on their ability to improve recognition. We analyze the relative performance of the algorithms on the AR face and ALOI datasets using both a SIFT-based recognition system and a simple pixel-based approach. Quantitative results demonstrate that color constancy methods can significantly improve classification accuracy. We also evaluate the approaches on the Caltech-101 dataset to determine how these algorithms affect performance under relatively normal illumination conditions.

Illumination changes and their effects on scene appearance pose serious problems to many computer vision algorithms. In this paper, we present the benefits that a chromaticity-based approach can provide to illumination compensation. We consider three computationally inexpensive illumination models, and demonstrate that customizing these models for chromatically dissimilar regions reduces mean absolute difference (MAD) error by 70% to 80% over computing the models globally for the entire image. We demonstrate that models computed for a given color are somewhat effective for different colors with similar hues (increasing MAD error by a factor of 6), but are ineffective for colors with dissimilar hues (increasing MAD error by a factor of 15). Finally, we find that model choice is less important if the model is customized for chromatically dissimilar regions. Effects of webcamera drivers are considered.

This paper proposes a cluster-to-cluster image color transform algorithm. Suntory Flowers announced the development of world’s first blue rose “APPLAUSE”. Since roses lack the blue pigment, it was long believed to be impossible. The key to success lies in the introduction of blue gene from pansy into rose. In the previous paper, PCA matching model was successfully applied to a seasonal color change in flowers, though it’s not real but virtual. However, the tonal color transitions between the different color hues such as red rose and blue pansy were not so smooth but unnatural because of spatially independent color blending. In addition, the clear separation of blue or purple petal colors from greenish backgrounds is not always easy too. The paper improves the color transform algorithm in the two points, firstly, the clear color separation by introducing a “complex log” color space and secondly, the smoothed tonal color transition by introducing a “time-variant” matrix for PCA matching. The proposed algorithm is applied to ROI (Region Of Interest) image color transform, for example, a blue rose creation from red rose by continuous color stealing of pansy blue.

SIFT is one of the most popular algorithms to extract points of interest from images. It is a scale+rotation invariant method. As a consequence, if one compares points of interest between two images subject to a large viewpoint change, then only a few, if any, common points will be retrieved. This may lead subsequent algorithms to failure, especially when considering structure and motion or object recognition problems. Reaching at least affine invariance is crucial for reliable point correspondences. Successful approaches have been recently proposed by several authors to strengthen scale+rotation invariance into affine invariance, using viewpoint simulation (

e.g.

the ASIFT algorithm). However, almost all resulting algorithms fail in presence of repeated patterns, which are common in man-made environments, because of the so-called perceptual aliasing. Focusing on ASIFT, we show how to overcome the perceptual aliasing problem. To the best of our knowledge, the resulting algorithm performs better than any existing generic point matching procedure.

In this article, we present a fast pedestrian detection system for driving assistance. We use current state-of-the-art HOG and LBP features and combine them into a set of powerful classifiers. We propose an encoding scheme that enables LBP to be used efficiently with the integral image approach. This way, HOG and LBP block features can be computed in constant time, regardless of block position or scale. To further speed up the detection process, a coarse-to-fine scanning strategy based on input resolution is employed. The original camera resolution is consecutively downsampled and fed to different stage classifiers. Early stages in low resolutions reject most of the negative candidate regions, while few samples are passed through all stages and are evaluated by more complex features. Results presented on the INRIA set show competetive accuracy performance, while both processing and training time of our system outperforms current state-of-the-art work.

For an effective vision-based HCI system, inference from natural means of sources (i.e. hand) is a crucial challenge in unconstrained environment. In this paper, we have aimed to build an interaction system through hand posture recognition for static finger spelling American Sign Language (ASL) alphabets and numbers. Unlike the interaction system based on speech, the coarticulation due to hand shape, position and movement influences the different aspects of sign language recognition. Due to this, we have computed the features which are invariant to translation, rotation and scaling. Considering these aspects as the main objectives of this research, we have proposed a three-step approach: first, features vector are computed using two moment based approaches namely Hu-Moment along with geometrical features and Zernike moment. Second, the categorization of symbols according to the fingertip is performed to avoid mis-classification among the symbols. Third, the extracted set of two features vectors (i.e. Hu-Moment with geometrical features and Zernike moment) are trained by Support Vector Machines (SVM) for the classification of the symbols. Experimental results of the proposed approaches achieve recognition rate of 98.5% using Hu-Moment with geometrical features and 96.2% recognition rate using Zernike moment for ASL alphabets and numbers demonstrating the dominating performance of Hu-Moment with geometrical features over Zernike moments.

In this paper, the well-known SIFT detector is extended with a bivariate feature localization. This is done by using function models that assume a Gaussian feature shape for the detected features. As function models we propose (a) a bivariate Gaussian and (b) a Difference of Gaussians. The proposed detector has all properties of SIFT, but provides invariance to affine transformations and blurring. It shows superior performance for strong viewpoint changes compared to the original SIFT. Compared to the most accurate affine invariant detectors, it provides competitive results for the standard test scenarios while performing superior in case of motion blur in video sequences.

Past work on Linear Dimensionality Reduction (LDR) has emphasized the issues of classification and dimension estimation. However, relatively less attention has been given to the critical issue of eigenvector selection. The main trend in feature extraction has been representing the data in a lower dimensional space, for example, using principal component analysis (PCA) without using an effective scheme to select an appropriate set of features/eigenvectors in this space. This paper addresses Linear Dimensionality Reduction through Eigenvector selection for object recognition. It has two main contributions. First, we propose a unified framework for one transform based LDR. Second, we propose a framework for two transform based DLR. As a case study, we consider PCA and Linear Discriminant Analysis (LDA) for the linear transforms. We have tested our proposed frameworks on several public benchmark data sets. Experiments on ORL, UMIST, and YALE Face Databases and MNIST Handwritten Digit Database show significant performance improvements in recognition that are based on eigenvector selection.

This paper describes a method to minimize the immense training time of the conventional Adaboost learning algorithm in object detection by reducing the sampling area. A new algorithm with respect to the geometric and accordingly the symmetric relations of the analyzed object is presented. Symmetry enhanced Adaboost (SEAdaboost) can limit the scanning area enormously, depending on the degree of the objects symmetry, while it maintains the detection rate. SEAdaboost allows to take advantage of the symmetric characteristics of an object by concentrating on corresponding symmetry features during the detection of weak classifiers. In our experiments we gain 39% reduced training time (in average) with slightly increasing detection rates (up to 2.4% and up to 6% depending on the object class) compared to the conventional Adaboost algorithm.

Occluding contour

(OC) plays important roles in many computer vision tasks. The study of using OC for visual inference tasks is however limited, partially due to the lack of robust OC acquisition technologies. In this work, benefit from a novel OC computation system, we propose applying OC information to category classification tasks. Specifically, given an image and its estimated occluding contours, we first compute a distance map with regard to the OCs. This map is then used to filter out distracting information in the image. The results are combined with standard recognition methods, bag-of-visual-words in our experiments, for category classification. In addition to the approach, we also present two OC datasets, which to the best of our knowledge are the first publicly available ones. The proposed method is evaluated on both datasets for category classification tasks. In all experiments, the proposed method significantly improves classification performances by about 10 percent.

Visualization of high-dimensional data is difficult to realize and manipulate with 2D display. For example, visualizing time-varying volume data (4D) with volume rendering and animation has spatial and temporal shielding, and data of 5 or more dimensions cannot be visualized on 2D display with existing methods. In this paper, we propose a method that expands high-dimensional data onto a 2D image plane. The proposed method uses the self-similarity of the fractal shape and achieves multi-scale high-dimensional data visualization on 2D display. With this method, we can visualize the entire domain of high-dimensional data without occlusions. Also, one-to-one correspondence in the elements of high-dimensional data and its 2D expansion enables us to manipulate high-dimensional data with 2D expanded result as an interface.

We extend our previous work on the exploration of static metabolic networks to evolving, and therefore dynamic, pathways. We apply our visualization software to data from a simulation of early metabolism. Thereby, we show that our technique allows us to test and argue for or against different scenarios for the evolution of metabolic pathways. This supports a profound and efficient analysis of the structure and properties of the generated metabolic networks and its underlying components, while giving the user a vivid impression of the dynamics of the system. The analysis process is inspired by Ben Shneiderman’s mantra of information visualization. For the overview, user-defined diagrams give insight into topological changes of the graph as well as changes in the attribute set associated with the participating enzymes, substances and reactions. This way, “interesting features” in time as well as in space can be recognized. A linked view implementation enables the navigation into more detailed layers of perspective for in-depth analysis of individual network configurations.

Interpolating diffusion tensor fields is a key technique to visualize the continuous behaviors of biological tissues such as nerves and muscle fibers. However, this has been still a challenging task due to the difficulty to handle possible degeneracy, which means the rotational inconsistency caused by degenerate points. This paper presents an approach to interpolating 3D diffusion tensors in 2D planar domains by aggressively locating the possible degeneracy while fully respecting the underlying transition of tensor anisotropy. The primary idea behind this approach is to identify the degeneracy using minimum spanning tree-based clustering algorithm, and resolve the degeneracy by optimizing the associated rotational transformations. Degenerate lines are generated in this process to retain the smooth transitions of anisotropic features. Comparisons with existing interpolation schemes will be also provided to demonstrate the technical advantages of the proposed approach.

We introduce Indented Pixel Tree Plots (IPTPs): a novel pixel-based visualization technique for depicting large hierarchies. It is inspired by the visual metaphor of indented outlines, omnipresent in graphical file browsers and pretty printing of source code. Inner vertices are represented as vertically arranged lines and leaf groups as horizontally arranged lines. A recursive layout algorithm places parent nodes to the left side of their underlying tree structure and leaves of each subtree grouped to the rightmost position. Edges are represented only implicitly by the vertically and horizontally aligned structure of the plot, leading to a sparse and redundant-free visual representation. We conducted a user study with 30 subjects in that we compared IPTPs and node-link diagrams as a within-subjects variable. The study indicates that working with IPTPs can be learned in less than 10 minutes. Moreover, IPTPs are as effective as node-link diagrams for accuracy and completion time for three typical tasks; participants generally preferred IPTPs. We demonstrate the usefulness of IPTPs by understanding hierarchical features of huge trees such as the NCBI taxonomy with more than 300,000 nodes.

Currently, visualization tools for large ontologies (e.g., pathway and gene ontologies) result in a very flat wide tree that is difficult to fit on a single display. This paper develops the concept of using an enhanced radial space-filling (ERSF) layout to show biological ontologies efficiently. The ERSF technique represents ontology terms as circular regions in 3D. Orbital connections in a third dimension correspond to non-tree edges in the ontology that exist when an ontology term belongs to multiple categories. Biologists can use the ERSF layout to identify highly activated pathway or gene ontology categories by mapping experimental statistics such as coefficient of variation and overrepresentation values onto the visualization. This paper illustrates the use of the ERSF layout to explore pathway and gene ontologies using a gene expression dataset from

E. coli

.

We propose a new method for the visualization of spectral images. It involves a perception-based spectrum segmentation using an adaptable thresholding of the stretched CIE standard observer color-matching functions. This allows for an underlying removal of irrelevant channels, and, consequently, an alleviation of the computational burden of further processings. Principal Components Analysis is then used in each of the three segments to extract the Red, Green and Blue primaries for final visualization. A comparison framework using two different datasets shows the efficiency of the proposed method.

In this paper we extend the classical marching cubes algorithm in computer graphics for isosurface polygonisation to make use of new developments in the sparse field level set method, which allows localised updates to the implicit level set surface. This is then applied to an example medical image analysis and visualisation problem, using user-guided intelligent agent swarms to correct holes in the surface of a brain cortex, where level set segmentation has failed to reconstruct the local surface geometry correctly from a magnetic resonance image. The segmentation system is real-time and fully interactive.

We propose a novel Multiple Instance Learning (MIL) framework to perform target localization from image sequences. The proposed approach consists of a softmax logistic regression MIL algorithm using log covariance features to automatically learn the model of a target that persists across input frames. The approach makes no assumptions about the target’s motion model and can be used to learn models for multiple targets present in the scene. The learned target models can also be updated in an online manner. We demonstrate the validity and usefulness of the proposed approach to localize targets in various scenes using commercial-grade surveillance cameras. We also demonstrate its applicability to bootstrap conventional tracking systems and show that automatic initialization using our technique helps to achieve superior performance.

Dealing with multi-object tracking in a particle filter raises several issues. A first essential point is to model possible interactions between objects. In this article, we represent these interactions using a fuzzy formalism, which allows us to easily model spatial constraints between objects, in a general and formal way. The second issue addressed in this work concerns the practical application of a multi-object tracking with a particle filter. To avoid a decrease of performances, a partitioned sampling method can be employed. However, to achieve good tracking performances, the estimation process requires to know the ordering sequence in which the objects are treated. This problem is solved by introducing, as a second contribution, a ranked partitioned sampling, which aims at estimating both the ordering sequence and the joint state of the objects. Finally, we show the benefit of our two contributions in comparison to classical approaches through two multi-object tracking experiments and the tracking of an articulated object.

We introduce a new method for integrated tracking and segmentation of a single non-rigid object in an monocular video, captured by a possibly moving camera. A closed-loop interaction between EM-like color-histogram-based tracking and Random Walker-based image segmentation is proposed, which results in reduced tracking drifts and in fine object segmentation. More specifically, pixel-wise spatial and color image cues are fused using Bayesian inference to guide object segmentation. The spatial properties and the appearance of the segmented objects are exploited to initialize the tracking algorithm in the next step, closing the loop between tracking and segmentation. As confirmed by experimental results on a variety of image sequences, the proposed approach efficiently tracks and segments previously unseen objects of varying appearance and shape, under challenging environmental conditions.

Motion estimation is one of the most important tasks in computer vision. One popular technique for computing dense motion fields consists in defining a large enough set of candidate motion vectors, and assigning one of such vectors to each pixel, so that a given cost function is minimized. In this work we propose a novel method for finding a small set of adequate candidates, making the minimization process computationally more efficient. Based on this method, we present algorithms for the estimation of dense optical flow using two minimization approaches: one based on a classic block-matching procedure, and another one based on entropy-controlled quadratic Markov measure fields which allow one to obtain smooth motion fields. Finally, we present the results obtained from the application of these algorithms to examples taken from the Middlebury database.

Motion estimation in image sequences is a fundamental problem for digital video coding. In this paper, we present a new approach for conservative motion estimation from multi-image sequences. We deal with a system in which most of the motions in the scene are conservative or near-conservative in a certain temporal interval with multi-image sequences. Then a single conservative velocity field in this temporal range can across several successive frames. This system can be proved to be fully constrained or over-constrained when the number of frames is greater than two. A framework with displaced frame difference (DFD) equations, spatial velocity modeling, a nonlinear least-squares model, and Gauss-Newton and Levenberg-Marguardt algorithms for solving the nonlinear system is developed. The proposed algorithm is evaluated experimentally with two standard test image sequences. All successive frames except the last one (used for reference frame) in this conservative system can be synthesized by the motion-compensated prediction and interpolation based on the estimated motion field. This framework can estimate large scale motion field that across more than two successive frames if most of the motions in the scene in the temporal interval are conservative or near-conservative and has better performance than the block matching algorithm.

To enable the precise detection of persons walking or running on the ground using unmanned Micro Aerial Vehicles (MAVs), we present the evaluation of the MCT algorithm based on intensity as well as gradient images for optical flow, focusing on accuracy as well as low computational complexity to enable the real-time implementation in light-weight embedded systems. Therefore, we give a detailed analysis of this algorithm on four optical flow datasets from the Middlebury database and show the algorithm’s performance when compared to other optical flow algorithms. Furthermore, different approaches for sub-pixel refinement and occlusion detection are discussed.

We propose a depth assisted video object tracking algorithm that utilizes a stereo vision technique to detect and handle various types of occlusions. The foreground objects are detected by using a depth and motion-based segmentation method. The occlusion detection is achieved by combining the depth segmentation results with the previous occlusion status of each track. According to the occlusion analysis results, different object correspondence algorithms are employed to track objects under various occlusions. The silhouette-based local best matching method deals with severe and complete occlusions without assumptions of constant movement and limited maximum duration. Experimental results demonstrate that the proposed system can accurately track multiple objects in complex scenes and provides improvements on dealing with different partial and severe occlusion situations.

An experimental analysis of three acquisition scenarios for face recognition at a distance is reported, namely: close, medium, and far distance between camera and query face, the three of them considering templates enrolled in controlled conditions. These three representative scenarios are studied using data from the NIST Multiple Biometric Grand Challenge, as the first step in order to understand the main variability factors that affect face recognition at a distance based on realistic yet workable and widely available data. The scenario analysis is conducted quantitatively in two ways. First, we analyze the information content in segmented faces in the different scenarios. Second, we analyze the performance across scenarios of three matchers, one commercial, and two other standard approaches using popular features (PCA and DCT) and matchers (SVM and GMM). The results show to what extent the acquisition setup impacts on the verification performance of face recognition at a distance.

Iris recognition from surveillance-type imagery is an active research topic in biometrics. However, iris identification in unconstrained conditions raises many proplems related to localization and alignment, and typically leads to degraded recognition rates. While development has mainly focused on more robust preprocessing, this work highlights the possibility to account for distortions at matching stage. We propose a constrained version of the Levenshtein Distance (LD) for matching of binary iris-codes as an alternative to the widely accepted Hamming Distance (HD) to account for iris texture distortions by e.g. segmentation errors or pupil dilation. Constrained LD will be shown to outperform HD-based matching on CASIA (third version) and ICE (2005 edition) datasets. By introducing LD alignment constraints, the matching problem can be solved in

O

(

n

·

s

) time and

O

(

n

 + 

s

) space with

n

and

s

being the number of bits and shifts, respectively.

We present a fuzzy multiscale segmentation algorithm aimed at hand images acquired by a mobile device, for biometric purposes. This algorithm is quasi-linear with the size of the image and introduces a stopping criterion that takes into account the texture of the regions and controls the level of coarsening. The algorithm yields promising results in terms of accuracy segmentation, having been compared to other well-known methods. Furthermore, its procedure is suitable for a posterior mobile implementation.

This paper reports an experimental analysis of footsteps as a biometric. The focus here is on information extracted from the time domain of signals collected from an array of piezoelectric sensors. Results are related to the largest footstep database collected to date, with almost 20,000 valid footstep signals and more than 120 persons, which is well beyond previous related databases. Three feature approaches have been extracted, the popular ground reaction force (GRF), the spatial average and the upper and lower contours of the pressure signals. Experimental work is based on a verification mode with a holistic approach based on PCA and SVM, achieving results in the range of 5 to 15% EER depending on the experimental conditions of quantity of data used in the reference models.

One of the most recent trends in biometrics is recognition by ear appearance in head profile images. Determining the region of interest which contains the ear is an important step in an ear biometric system. To this end, we propose a robust, simple and effective method for ear detection from profile images by employing a bank of curved and stretched Gabor wavelets, known as banana wavelets. A 100% detection rate is achieved here on a group of 252 profile images from XM2VTS database. The banana wavelets technique demonstrates better performances than Gabor wavelets technique. This indicates that the curved wavelets are advantageous here. Also the banana wavelet technique is applied to a new and more challenging database which highlights practical considerations of a more realistic deployment. This ear detection technique is fully automated, has encouraging performance and appears to be robust to degradation by noise.

Manifold learning is a novel approach in non-linear dimensionality reduction that has shown great potential in numerous applications and has gained ground compared to linear techniques. In addition, sparse representations have been recently applied on computer vision problems with success, demonstrating promising results with respect to robustness in challenging scenarios. A key concept shared by both approaches is the notion of sparsity. In this paper we investigate how the framework of sparse representations can be applied in various stages of manifold learning. We explore the use of sparse representations in two major components of manifold learning: construction of the weight matrix and classification of test data. In addition, we investigate the benefits that are offered by introducing a weighting scheme on the sparse representations framework via the weighted LASSO algorithm. The underlying manifold learning approach is based on the recently proposed spectral regression framework that offers significant benefits compared to previously proposed manifold learning techniques. We present experimental results on these techniques in three challenging face recognition datasets.

In this paper, we present a novel graph, sub-graph and super-graph based face representation which captures the facial shape changes and deformations caused due to pose changes and use it in the construction of an adaptive appearance model. This work is an extension of our previous work proposed in [1]. A sub-graph and super-graph is extracted for each pair of training graphs of an individual and added to the graph model set and used in the construction of appearance model. The spatial properties of the feature points are effectively captured using the graph model set. The adaptive graph appearance model constructed using the graph model set captures the temporal characteristics of the video frames by adapting the model with the results of recognition from each frame during the testing stage. The graph model set and the adaptive appearance model are used in the two stage matching process, and are updated with the sub-graphs and super-graphs constructed using the graph of the previous frame and the training graphs of an individual. The results indicate that the performance of the system is improved by using sub-graphs and super-graphs in the appearance model.

The estimation of left ventricle motion and deformation from series of images has been an area of attention in the medical image analysis and still remains an open and challenging research problem. The proper tracking of left ventricle wall can contribute to isolate the location and extent of ischemic or infarcted myocardium. This work describes a method to automatically estimate the displacement fields for a beating heart based on the study of the variation in the frequency content of a non-stationary image as time varies. Results obtained with this automated method in synthetic images are compared with traditional gradient based method. Furthermore, experiments involving cardiac SPECT images are also presented.

In this paper, we present a graph-based multi-resolution approach for mitosis extraction in breast cancer histological whole slide images. The proposed segmentation uses a multi-resolution approach which reproduces the slide examination done by a pathologist. Each resolution level is analyzed with a focus of attention resulting from a coarser resolution level analysis. At each resolution level, a spatial refinement by semi-supervised clustering is performed to obtain more accurate segmentation around edges. The proposed segmentation is fully unsupervised by using domain specific knowledge.

We have performed segmentation procedures on a large number of images from two mammalian cell lines that were seeded at low density, in order to study trends in the segmentation results and make predictions about cellular features that affect segmentation accuracy. By comparing segmentation results from approximately 40000 cells, we find a linear relationship between the highest segmentation accuracy seen for a given cell and the fraction of pixels in the neighborhood of the edge of that cell. This fraction of pixels is at greatest risk for error when cells are segmented. We call the ratio of the size of this pixel fraction to the size of the cell the extended edge neighborhood and this metric can predict segmentation accuracy of any isolated cell.

We introduce variational optical flow computation involving the prior with the fractional order differentiations. The fractional order differentiation is a typical tool in signal processing and image analysis. The zero crossing of a fractional order Laplacian yields a good performance for edge detection. As a sequel of edge detection with the fractional order differentiations, we deal with variational optical flow computation involving the fractional order differentiations on optical flow vectors. The method allows us to detect discontinuity of optical flow using linear operations.

In the field of assisted reproductive technologies,

ICSI

fertilization is a medically-assisted reproduction technique, enabling infertile couples to achieve successful pregnancy. In this field crucial points are: the analysis of clinical data of the patient, aimed at adopting an appropriate stimulation protocol to obtain an adequate number of oocytes, and the selection of the best oocytes to fertilize. In this paper we would provide a framework able to extract useful morphological features from oocyte images that combined with the provided clinical data of the patients can be used to discover new information for defining therapeutic plans for new patients as well as selecting the most promising oocytes.

Physically-based image synthesis requires measured spectral quantities for illuminants and reflectances as part of the virtual scene description to compute trustworthy lighting simulations. When spectral distributions are not available, a method to reconstruct spectra from color triplets needs to be applied. A comprehensive evaluation of the practical applicability of previously published approaches in the context of realistic rendering is still lacking. Thus, we designed three different comparison scenarios typical for computer graphic applications to evaluate the suitability of the methods to reconstruct illumination and reflectance spectra. Furthermore, we propose a novel approach applying empirical mean spectra as basis functions to reconstruct spectral distributions. The mean spectra are derived from averaging sets of typical red, green, and blue spectra. This method is intuitive, computationally inexpensive, and achieved the best results for all scenarios in our evaluation. However, reconstructed spectra are not unrestrictedly applicable in physically-based rendering where reliable synthetic images are crucial.

Consider any real structure that can be modeled by a set of straight line segments. This can be a network of streets in a city, tunnels in a mine, corridors in a building, pipes in a factory, etc. We want to approximate a minimal number of locations where to place “guards” (either men or machines), in a way that any point of the network can be “seen” by at least one guard. A guard can see all points on segments it is on (and nothing more). As the problem is known to be NP-hard, we consider three greedy-type algorithms for finding approximate solutions. We show that for each of these, theoretically the ratio of the approximate to the optimal solution can increase without bound with the increase of the number of segments. Nevertheless, our extensive experiments show that on randomly generated instances, the approximate solutions are

always

very close to the optimal ones and often are, in fact, optimal.

This paper describes a method for Automatic hole characterization on 3D meshes, avoiding user intervention to decide which regions of the surface should be corrected. The aim of the method is to classify real and false anomalies without user intervention by using a contours irregularity measure based on two geometrical estimations: the torsion contour’s estimation uncertainty, and an approximation of geometrical shape measure surrounding the hole.

Direction fields are an essential ingredient in controlling surface appearance for applications ranging from anisotropic shading to texture synthesis and non-photorealistic rendering. Applying local principal covariance analysis, we present a simplistic way for constructing local frames used for vector field generation on point-sampled models. Different kinds of vector fields can be achieved by assigning different planar vectors to the local coordinates. Unlike previous methods, in the proposed algorithm, there is no need of user constraints or any extra smoothing and relaxation process. Experimental results in the isotropic remeshing and texture synthesis are used to demonstrate its performance.

We present an approach for real-time rendering of complex 3D scenes consisting of millions of polygons on limited graphics hardware. In a preprocessing step, powerful hardware is used to gain fine granular global visibility information of a scene using an adaptive sampling algorithm. Additively the visual influence of each object on the eventual rendered image is estimated. This influence is used to select the most important objects to display in our approximative culling algorithm. After the visibility data is compressed to meet the storage capabilities of small devices, we achieve an interactive walkthrough of the Power Plant scene on a standard netbook with an integrated graphics chipset.

Mesh editing is a time-consuming and error prone process when changes must be manually applied to repeated structures in the mesh. Since mesh design is a major bottleneck in the creation of computer games and animation, simplifying the process of mesh editing is an important problem. We propose a fast and accurate method for performing region matching which is based on the manifold harmonics transform. We then demonstrate this matching method in the context of

nonlocal mesh editing

- propagating mesh editing operations from a single source region to multiple target regions which may be arbitrarily far away. This contribution will lead to more efficient methods of mesh editing and character design.

Multi-view range image integration aims at producing a single reasonable 3D point cloud. The point cloud is likely to be inconsistent with the measurements topologically and geometrically due to registration errors and scanning noise. This paper proposes a novel integration method cast in the framework of Markov random fields (MRF). We define a probabilistic description of a MRF model designed to represent not only the interpoint Euclidean distances but also the surface topology and neighbourhood consistency intrinsically embedded in a predefined neighbourhood. Subject to this model, points are clustered in aN iterative manner, which compensates the errors caused by poor registration and scanning noise. The integration is thus robust and experiments show the superiority of our MRF-based approach over existing methods.

This paper presents a novel scheme for automatically aligning two widely separated 3D scenes via the use of viewpoint invariant features. The key idea of the proposed method is following. First, a number of dominant planes are extracted in the SfM 3D point cloud using a novel method integrating RANSAC and MDL to describe the underlying 3D geometry in urban settings. With respect to the extracted 3D planes, the original camera viewing directions are rectified to form the front-parallel views of the scene. Viewpoint invariant features are extracted on the canonical views to provide a basis for further matching. Compared to the conventional 2D feature detectors (e.g. SIFT, MSER), the resulting features have following advantages: (1) they are very discriminative and robust to perspective distortions and viewpoint changes due to exploiting scene structure; (2) the features contain useful local patch information which allow for efficient feature matching. Using the novel viewpoint invariant features, wide-baseline 3D scenes are automatically aligned in terms of robust image matching. The performance of the proposed method is comprehensively evaluated in our experiments. It’s demonstrated that 2D image feature matching can be significantly improved by considering 3D scene structure.

In this paper, we present an algorithm for estimating disparity for images containing large textureless regions. We propose a fast and efficient region growing algorithm for estimating the stereo disparity. Though we present results on ice images, the algorithm can be easily used for other applications. We modify the first-best region growing algorithm using relaxed uniqueness constraints and matching for sub-pixel values and slant surfaces. We provide an efficient method for matching multiple windows using a linear transform. We estimate the parameters required by the algorithm automatically based on initial correspondences. Our method was tested on synthetic, benchmark and real outdoor data. We quantitatively demonstrated that our method performs well in all three cases.

A novel method of synthetic shape reconstruction from photometric stereo data combined with the FF-based method is presented, aiming at obtaining more accurate shape. First, a shape is reconstructed from color images using a modified FF-based algorithm. Then, with the shape as initial value, a more accurate shape is synthetically reconstructed based on the Jacobi iterative method. The synthesis is realized as follows: the reconstruction is sequentially made in each of small image subareas, using the depths in the neighboring subareas as boundary values, which is iterated until the overall shape converges. The division to image subareas enables us to synthesize large shapes.

Generating accurate three dimensional planetary models is becoming increasingly important as NASA plans manned missions to return to the Moon in the next decade. This paper describes a 3D surface and albedo reconstruction from orbital imagery. The techniques described here allow us to automatically produce seamless, highly accurate digital elevation and albedo models from multiple stereo image pairs while significantly reducing the influence of image noise. Our technique is demonstrated on the entire set of orbital images retrieved by the Apollo 15 mission.

Unmanned Aircraft Systems (UAS) have been used in many military and civil applications, particularly surveillance. One of the best ways to use the capacity of a UAS imaging system is by constructing a mosaic of the recorded video. This paper presents a novel algorithm for the construction of superresolution mosaicking. The algorithm is based on the Levenberg Marquardt (LM) method. Hubert prior is used together with four different cliques to deal with the ill-conditioned inverse problem and to preserve edges. Furthermore, the Lagrange multiplier is compute without using sparse matrices. We present the results with synthetic and real UAS surveillance data, resulting in a great improvement of the visual resolution. For the case of synthetic images, we obtained a PSNR of 47.0 dB, as well as a significant increase in the details visible for the case of real UAS frames in only ten iterations.

Hand dyeing generates artistic representations with unique and complex patterns. The aesthetics of dyed patterns on a cloth originate from the physical properties of dyeing in the cloth and the geometric operations of the cloth. Although many artistic representations have been studied in the field of non-photorealistic rendering, dyeing remains a challenging and attractive topic. In this paper, we propose a new framework for simulating dyeing techniques that considers the geometry of the folded cloth. Our simulation framework of dyeing in folded woven cloth is based on a novel dye transfer model that considers diffusion, adsorption, and supply. The dye transfer model is discretized on a 3D graph to approximate the folded woven cloth designed by user interactions. We also develop new methods for dip dyeing and tie-dyeing effects. Comparisons of our simulated results with real dyeing demonstrate that our simulation is capable of representing characteristics of dyeing.

We investigate menu distance, size, and related techniques to understand and optimize menu performance in VR. We show how user interaction using ray casting and Pointer-Attached-to-Menu (PAM) pointing techniques is affected by menu size and distance from users. Results show how selection angle – an angle to targets that depends on menu size and distance – relates to selection times. Mainly, increasing selection angle lowers selection time. Maintaining a constant selection angle, by a technique called “auto-scale”, mitigates distance effects for ray casting. For small menus, PAM appears to perform as well as or potentially faster than ray casting. Unlike standard ray casting, PAM is potentially useful for tracked game controllers with restricted DOF, relative-only tracking, or lower accuracy.

Visual and haptic factors can affect a user’s interpretation of vibrotactile cues communicating location of objects in a real or virtual environment. Identifying and understanding relevant factors will lead to better device and interface design, for example, through procedures that adjust for systematic error or per-user differences. We considered direct effects of hand-tactor contact geometry and a possible cross-modal effect of the visual interface. Our experiment examined contact geometry on a single row of tactors and presence of a visual border on a graphical region that mapped to the tactor array. We measured the relationship between vibrotactile array stimulus coordinates and user responses. Contact geometry that emphasized a certain tactor increased tendency for subjects to mark near it. Effects of visual borders were noticeable but subtle, acting more as a modulating factor.

Several existing procedural modeling systems are able to generate large 3D models of cities. However, none of these systems can automatically create 3D models of freeways and freeway interchanges, even though these are important features in 3D urban landscape. We have implemented a system that automatically creates 3D models of road surfaces, bridges, tunnels, freeways, and freeway interchanges, taking user design and preferences into account. While we allow the graphics designers to control the positions of the control points according to their aesthetic appeal, our system automatically generates 3D models of road surfaces and freeway connector ramps that are properly smoothed, banked and connected.

In this paper, we propose an automatic learning method for gesture recognition. We combine two different pattern recognition techniques: the Self-Organizing Map (SOM) and Support Vector Machine (SVM). First, we apply the SOM to divide the sample data into phases and construct a state machine. Next, we apply the SVM to learn the transition conditions between nodes. An independent SVM is constructed for each node. Of the various pattern recognition techniques for multi-dimensional data, the SOM is suitable for categorizing data into groups, and thus it is used in the first process. On the other hand, the SVM is suitable for partitioning the feature space into regions belonging to each class, and thus it is used in the second process. Our approach is unique and effective for multi-dimensional and time-varying gesture recognition. The proposed method is a general gesture recognition method that can handle any kinds of input data from any input device. In the experiment presented in this paper, we used two Nintendo Wii Remote controllers, with three-dimensional acceleration sensors, as input devices. The proposed method successfully learned the recognition models of several gestures.