Graphics Recognition. Achievements, Challenges, and Evolution

Rulings are graphical primitives that are essential for document structure recognition. However in the case of ancient documents, bad printing techniques or bad conditions of conservation induce problems for their efficient recognition. Consequently, usual line segment extractors are not powerful enough to properly extract all the rulings of a heterogeneous document. In this paper, we propose a new method for ruling recognition, based on perceptive vision: we show that combining several levels of vision improves ruling recognition. Thus, it is possible to put forward hypothesis on the nature of the rulings at a given resolution, and to confirm or infirm their presence and find their exact position at higher resolutions.

We propose an original strategy of cooperation between resolutions and present tools to set up a correspondence between the elements extracted at each resolution. We validate this approach on images of ancient newspaper pages (dated between 1848 and 1944). We also propose to use the extracted rulings for the structure analysis of newspaper pages. We show that using more reliable extracted rulings simplifies and improves document structure recognition.

The motivation behind our work is to present a new methodology for symbol recognition. The proposed method employs a structural approach for representing visual associations in symbols and a statistical classifier for recognition. We vectorize a graphic symbol, encode its topological and geometrical information by an attributed relational graph and compute a signature from this structural graph. We have addressed the sensitivity of structural representations to noise, by using data adapted fuzzy intervals. The joint probability distribution of signatures is encoded by a Bayesian network, which serves as a mechanism for pruning irrelevant features and choosing a subset of interesting features from structural signatures of underlying symbol set. The Bayesian network is deployed in a supervised learning scenario for recognizing query symbols. The method has been evaluated for robustness against degradations & deformations on pre-segmented 2D linear architectural & electronic symbols from GREC databases, and for its recognition abilities on symbols with context noise i.e. cropped symbols.

Two hundred web tables from ten sites were imported into Excel. The tables were edited as needed, then converted into layout independent Wang Notation using the Table Abstraction Tool (TAT). The output generated by TAT consists of XML files to be used for constructing narrow-domain ontologies. On an average each table required 104 seconds for editing. Augmentations like aggregates, footnotes, table titles, captions, units and notes were also extracted in an average time of 93 seconds. Every user intervention was logged and audited. The logged interactions were analyzed to determine the relative influence of factors like table size, number of categories and various types of augmentations on the processing time. The analysis suggests which aspects of interactive table processing can be automated in the near term, and how much time such automation would save. The correlation coefficient between predicted and actual processing time was 0.66.

In this paper we evaluate four graph distance measures. The analysis is performed for document retrieval tasks. For this aim, different kind of documents are used including line drawings (symbols), ancient documents (ornamental letters), shapes and trademark-logos. The experimental results show that the performance of each graph distance measure depends on the kind of data and the graph representation technique.

In this paper we present a method to robustly detect circular arcs in a line drawing image. The method is fast, robust and very reliable, and is capable of assessing the quality of its detection. It is based on Random Sample Consensus minimization, and uses techniques that are inspired from object tracking in image sequences. It is based on simple initial guesses, either based on connected line segments, or on elementary mainstream arc detection algorithms. Our method consists of gradually deforming these circular arc candidates as to precisely fit onto the image strokes, or to reject them if the fitting is not possible, this virtually eliminates spurious detections on the one hand, and avoiding non-detections on the other hand.

The median-shift, a new clustering algorithm, is proposed to automatically identify the palette of colored graphics, a pre-requisite for graphics vectorization. The median-shift is an iterative process which shifts each data point to the “median” point of its neighborhood defined thanks to a distance measure and a maximum radius, the only parameter of the method. The process is viewed as a graph transformation which converges to a set of clusters made of one or several connected vertices. As the palette identification depends on color perception, the clustering is performed in the L*a*b* feature space. As pixels located on edges are made of mixed colors not expected to be part of the palette, they are removed from the initial data set by an automatic pre-processing. Results are shown on scanned maps and on the Macbeth color chart and compared to well established methods.

This paper presents a new technique for detection of digital circles and circular arcs using chord property and sagitta property. It is shown how a variant of the chord property of an Euclidean circle can be used to detect a digital circle or a circular arc. Based on this property, digital circular arcs are first extracted and then using the sagitta property, their centers and radii are computed. Several arcs are merged together to form a complete digital circle or a larger arc. Finally, a technique based on Hough transform is used to improve the accuracy of computing the centers and radii. Experimental results have been furnished to demonstrate the efficiency of the proposed method.

Understanding of graphic objects has become a problem of pertinence in today’s context of digital documentation and document digitization, since graphic information in a document image may be present in several forms, such as engineering drawings, architectural plans, musical scores, tables, charts, extended objects, hand-drawn sketches, etc. There exist quite a few approaches for segmentation of graphics from text, and also a separate set of techniques for recognizing a graphics and its characteristic features. This paper introduces a novel geometric algorithm that performs the task of segmenting out all the graphic objects in a document image and subsequently also works as a high-level tool to classify various graphic types. Given a document image, it performs the text-graphics segmentation by analyzing the geometric features of the minimum-area isothetic polygonal covers of all the objects for varying grid spacing, g. As the shape and size of a polygonal cover depends on g, and each isothetic polygon is represented by an ordered sequence of its vertices, the spatial relationship of the polygons corresponding to a higher grid spacing with those corresponding to a lower spacing, is used for graphics segmentation and subsequent classification. Experimental results demonstrate its efficiency, elegance, and versatility.

Raster maps are an important source of road information. Because of the overlapping map features (e.g., roads and text labels) and the varying image quality, extracting road vector data from raster maps usually requires significant user input to achieve accurate results. In this paper, we present an accurate road vectorization technique that minimizes user input by combining our previous work on extracting road pixels and road-intersection templates to extract accurate road vector data from raster maps. Our approach enables GIS applications to exploit the road information in raster maps for the areas where the road vector data are otherwise not easily accessible, such as the countries of the Middle East. We show that our approach requires minimal user input and achieves an average of 93.2% completeness and 95.6% correctness in an experiment using raster maps from various sources.

In this paper, we study the segmentation of sketched engineering drawings into a set of straight and curved segments. Our immediate objective is to produce a benchmarking method for segmentation algorithms. The criterion is to minimise the differences between what the algorithm detects and what human beings perceive. We have created a set of sketched drawings and have asked people to segment them. By analysis of the produced segmentations, we have obtained the number and locations of the segmentation points which people perceive. Evidence collected during our experiments supports useful hypotheses, for example that not all kinds of segmentation points are equally difficult to perceive. The resulting methodology can be repeated with other drawings to obtain a set of sketches and segmentation data which could be used as a benchmark for segmentation algorithms, to evaluate their capability to emulate human perception of sketches.

The design of a slide presentation is a creative process. In this process first, humans visualize in their minds what they want to explain. Then, they have to be able to represent this knowledge in an understandable way. There exists a lot of commercial software that allows to create our own slide presentations but the creativity of the user is rather limited. In this article we present an application that allows the user to create and visualize a slide presentation from a sketch. A slide may be seen as a graphical document or a diagram where its elements are placed in a particular spatial arrangement. To describe and recognize slides a syntactic approach is proposed. This approach is based on an Adjacency Grammar and a parsing methodology to cope with this kind of grammars. The experimental evaluation shows the performance of our methodology from a qualitative and a quantitative point of view. Six different slides containing different number of symbols, from 4 to 7, have been given to the users and they have drawn them without restrictions in the order of the elements. The quantitative results give an idea on how suitable is our methodology to describe and recognize the different elements in a slide.

This paper presents a new method to extract areas of interest in drop caps and particularly the most important shape: Letter itself. This method relies on a combination of a Aujol and Chambolle algorithm and a Segmentation using a Zipf Law and can be enhanced as a three-step process: 1)Decomposition in layers 2)Segmentation using a Zipf Law 3)Selection of the connected components.

Recently we have developed a model for shape description and matching. Based on minimum spanning trees construction and specifics stages like the mixture, it seems to have many desirable properties. Recognition invariance in front shift, rotated and noisy shape was checked through median scale tests related to GREC symbol reference database. Even if extracting the topology of a shape by mapping the shortest path connecting all the pixels seems to be powerful, the construction of graph induces an expensive algorithmic cost. In this article we discuss on the ways to reduce time computing. An alternative solution based on image compression concepts is provided and evaluated. The model no longer operates in the image space but in a compact space, namely the Discrete Cosine space. The use of block discrete cosine transform is discussed and justified. The experimental results led on the GREC2003 database show that the proposed method is characterized by a good discrimination power, a real robustness to noise with an acceptable time computing.

In this paper, we present a novel unifying concept of pairwise spatial relations. We develop two way directional relations with respect to a unique point set, based on topology of the studied objects and thus avoids problems related to erroneous choices of reference objects while preserving symmetry. The method is robust to any type of image configuration since the directional relations are topologically guided. An automatic prototype graphical symbol retrieval is presented in order to establish its expressiveness.

A common problem encountered in recognizing signs in real-scene images is the perspective deformation. In this paper, we employ a descriptor named Cross Ratio Spectrum for recognizing real scene signs. Particularly, this method will be applied in two different ways: recognizing a multi-component sign as an whole entity or recognizing individual components separately. For the second strategy, a graph matching is used to finally decide the identify of the query sign.

Interpretation of graphical document images is a challenging task as it requires proper understanding of text/graphics symbols present in such documents. Difficulties arise in graphical document recognition when text and symbol overlapped/touched. Intersection of text and symbols with graphical lines and curves occur frequently in graphical documents and hence separation of such symbols is very difficult.

Several pattern recognition and classification techniques exist to recognize isolated text/symbol. But, the touching/overlapping text and symbol recognition has not yet been dealt successfully. An interesting technique, Scale Invariant Feature Transform (SIFT), originally devised for object recognition can take care of overlapping problems. Even if SIFT features have emerged as a very powerful object descriptors, their employment in graphical documents context has not been investigated much. In this paper we present the adaptation of the SIFT approach in the context of text character localization (spotting) in graphical documents. We evaluate the applicability of this technique in such documents and discuss the scope of improvement by combining some state-of-the-art approaches.

The proceedings of many technical events in different areas of knowledge witness the history of the development of that area. LiveMemory is a user friendly tool developed to generate digital libraries of event proceedings. This paper describes the module designed to perform content recognition in LiveMemory.

A colour image segmentation (CIS) process for scanned historical maps is presented to overcome common problems associated with segmentation of old documents such as (1) variation in colour values of the same colour layer within one map page, (2) differences in typical colour values between homogeneous areas and thin line-work, which belong both to the same colour layer, and (3) extensive parameterization that results in a lack of robustness. The described approach is based on a two-stage colour layer prototype search using a constrained sampling design. Global colour layer prototypes for the identification of homogeneous regions are derived based on colour similarity to the most extreme colour layer values identified in the map page. These global colour layer prototypes are continuously adjusted using relative distances between prototype positions in colour space until a reliable sample is collected. Based on this sample colour layer seeds and directly connected neighbors of the same colour layer are determined resulting in the extraction of homogeneous colour layer regions. In the next step the global colour layer prototypes are recomputed using a new sample of colour values along the margins of identified homogeneous coloured regions. This sampling step derives representative prototypes of map layer sections that deviate significantly from homogeneous regions of the same layers due to bleaching, mixed or false colouring and ageing of the original scanned documents. A spatial expansion process uses these adjusted prototypes as start criterion to assign the remaining colour layer parts. The approach shows high robustness for map documents that suffer from low graphical quality indicating some potential for general applicability due to its simplicity and the limited need for preliminary information. The only input required is the colours and number of colour layers present in the map.

This paper presents a new comparative objective method for image quality evaluation. This method relies on two keys points: a local objective evaluation and a perceptual gathering. The local evaluation concerns the dissimilarities between the degraded image and the reference image; it is based on a gray-level local Hausdorff distance. This local Hausdorff distance uses a generalized distance transform which is studied here. The evaluation result is a local dissimilarity map (LDMap). In order to include perceptual information, a perceptual map based on the image properties is then proposed. The coefficients of this map are used to weight and to gather the LDMap measures into a single quality measure. The perceptual map is tunable and it gives encouraging quality measures even with naive parameters.

Empirical performance evaluation of raster to vector methods is an important topic in the area of graphics recognition. By studying automatic vectorization methods we can reveal the maturity of the tested methods whether as a research prototype or a commercial software. Arc Segmentation Contest held in conjunction with the eighth IAPR International Workshop on Graphics Recognition (GREC’09) is an excellent opportunity for researchers to present the results of their proposed raster to vector methods. The contest provides a uniform platform where the output of different methods can be analyzed. The relevance of the contest is further revealed by the creation of new test images with their ground truth data. Old documents were used in this contest. Five methods participated (two research prototypes and three commercial software). Two tests were performed namely between-methods test (participated by all methods) and within-method test (participated by only one method). This paper presents the results of the contest.

In this paper we present an algorithm for performance characterization of symbol localization systems. This algorithm is aimed to be a more “reliable” and “open” solution to characterize the performance. To achieve that, it exploits only single points as the result of localization and offers the possibility to reconsider the localization results provided by a system. We use the information about context in groundtruth, and overall localization results, to detect the ambiguous localization results. A probability score is computed for each matching between a localization point and a groundtruth region, depending on the spatial distribution of the other regions in the groundtruth. Final characterization is given with detection rate/probability score plots, describing the sets of possible interpretations of the localization results, according to a given confidence rate. We present experimentation details along with the results for the symbol localization system of [1], exploiting a synthetic dataset of architectural floorplans and electrical diagrams (composed of 200 images and 3861 symbols).

This paper tries to sum up the discussions held during the sessions of GREC’09, as well as at the final panel session. As it is always good to know where you are coming from, the paper briefly takes a look back at the discussions held two years earlier, before looking ahead at the future challenges for our research community. A number of points raised two years ago remain very much valid, but we also try to identify some new grand challenges for the field of graphics recognition.