CERMINE: automatic extraction of structured metadata from scientific literature

The purpose of the character extraction step is to extract individual characters from the PDF stream along with their positions on the page, widths and heights. These geometric parameters play important role in further steps, in particular page segmentation and content classification.

The implementation of character extraction is based on open-source iText [30] library. We use iText to iterate over PDF’s text-showing operators. During the iteration, we extract text strings along with their size and position on the page. Next, extracted strings are split into individual characters and their individual widths and positions are calculated. The result is an initial flat structure of the document, which consists only of pages and characters. The widths and heights computed for individual characters are approximate and can slightly differ from the exact values depending on the font, style and characters used. Fortunately, those approximate values are sufficient for further steps.

The goal of page segmentation step is to create a geometric hierarchical structure storing the document’s content. As a result the document is represented by a list of pages, each page contains a set of zones, each zone contains a set of text lines, each line contains a set of words, and finally each word contains a set of individual characters. Each object in the structure has its content, position and dimensions. The structure is heavily used in further steps, especially zone classification and bibliography extraction.

Page segmentation is implemented with the use of a bottom-up Docstrum algorithm [31]:

A few improvements were added to the Docstrum-based implementation of page segmentation:

A PDF file contains by design a stream of strings that undergoes extraction and segmentation process. As a result, we obtain pages containing characters grouped into zones, lines and words, all of which have a form of unsorted bag of items. The aim of setting the reading order is to determine the right sequence in which all the structure elements should be read. This information is used in zone classifiers and also allows to extract the full text of the document in the right order. An example document page with a reading order of the zones is shown in Fig. 4.

Reading order resolving algorithm is based on a bottom-up strategy: first characters are sorted within words and words within lines horizontally, then lines are sorted vertically within zones, and finally we sort zones. The fundamental principle for sorting zones was taken from [32]. We make use of an observation that the natural reading order in most modern languages descends from top to bottom, if successive zones are aligned vertically, otherwise it traverses from left to right. There are few exceptions to this rule, for example, Arabic script, and such cases would not be handled properly by the algorithm. This observation is reflected in the distances counted for all zone pairs: the distance is calculated using the angle of the slope of the vector connecting zones. As a result, zones aligned vertically are in general closer than those aligned horizontally. Then, using an algorithm similar to hierarchical clustering methods, we build a binary tree by repeatedly joining the closest zones and groups of zones. After that, for every node its children are swapped, if needed. Finally, an in order tree traversal gives the desired zones order.

The features used by the classifiers were selected with the use of the zone validation dataset (all the datasets used for experiments are described in Sect. 5.1). For each classifier, we analysed 97 features in total. The features capture various aspects of the content and surroundings of the zones and can be divided into the following categories:In general, feature selection was performed by analysing the correlations between the features and between features and expected labels. For simplicity, we treat all the features as numerical variables; the values of binary features are decoded as 0 or 1. The labels, on the other hand, are an unordered categorical variable.

Let L be a set of zone labels for a given classifier, n the number of the observations (zones) in the validation dataset and \(k = 97\) the initial number of analysed features. For ith feature, where \(0 \le i < k\), we can define \(f_i \in R^n\), a vector of the values of the feature ith for subsequent observations. Let also \(l \in L^n\) be the corresponding vector of zone labels.

In the first step, we removed redundant features, highly correlated with other features. For each pair of feature vectors, we calculated the Pearson’s correlation score and identified all the pairs \(f_i, f_j \in R^n\), such thatNext, for every feature from highly correlated pairs, we calculated the mean absolute correlation:and from each highly correlated pair, the feature with higher meanCorr was eliminated. This left us with 78 and 75 features for initial and metadata classifiers, respectively. Let’s denote the number of remaining features as \(k'\).

After eliminating features using correlations between them, we analysed the features using their associations with the expected zone labels vector l. To calculate the correlation between a single feature vector \(f_i\) (numeric) and label vector l (unordered categorical), we employed Goodman and Kruskal’s \(\tau \) (tau) measure [34]. Let’s denote it as \(\tau (f_i, l)\).

Let \(f_0, f_1, \ldots f_{k'-1}\) be the sequence of the feature vectors ordered by non-decreasing \(\tau \) measure, that isThe features were then added to the classifier one by one, starting from the best one (the mostly correlated with the labels vector, \(f_{k'-1}\)), and at the end the classifier contained the entire feature set. At each step, we performed a fivefold cross-validation on the validation dataset and calculated the overall F score as an average for individual labels. For completeness, we also repeated the same process with reversed order of the features, starting with less useful features. The results for initial and metadata classifier are shown in Figs. 5 and 6, respectively.

Using these results, we eliminated a number of the least useful features \(f_0, f_1, \ldots f_{t}\), such that the performance of the classifier with the remaining features was similar to the performance of the classifier trained on the entire feature set. Final feature sets contain 53 and 51 features for initial and metadata classifier, respectively.

SVM parameters were also estimated using the zone validation dataset. The feature vectors were scaled linearly to interval [0, 1] according to the bounds found in the learning samples. In order to find the best parameters for the classifiers we performed a grid search over a three-dimensional space \(\langle K, \varGamma , C \rangle \), where K is a set of kernel function types (linear, fourth degree polynomial, radial-basis and sigmoid), \(\varGamma = \{2^i | i \in [-15,3]\}\) is a set of possible values of the kernel coefficient \(\gamma \), and \(C = \{2^i | i \in [-5,15]\}\) is a set of possible values of the penalty parameter. For every combination of the parameters, we performed a fivefold cross-validation. Finally, we chose those parameters, for which we obtained the highest mean F score (calculated as an average for individual classes). We also used classes weights based on the number of their training samples to set larger penalty for less represented classes.

Parameters for the best obtained results are presented in Tables 3 and 4. In both cases, we chose radial-basis kernel function, and chosen values of C and \(\gamma \) parameters are \(2^5\) and \(2^{-3}\) in the case of initial classifier and \(2^9\) and \(2^{-3}\) in the case of metadata classifier.

References zones contain a list of reference strings, each of which can span over one or more text lines. The goal of reference strings extraction is to split the content of those zones into individual reference strings. This step utilizes unsupervised machine learning techniques, which allows to omit time-consuming training set preparation and learning phases, while achieving very good extraction results.

Every bibliographic reference is displayed in the PDF document as a sequence of one or more text lines. Each text line in a reference zone belongs to exactly one reference string, some of them are first lines of their reference, others are inner or last ones. The sequence of all text lines belonging to bibliography section can be represented by the following regular expression:

In order to group text lines into consecutive references, first we determine which lines are first lines of their references. A set of such lines is presented in Fig. 7. To achieve this, we transform all lines to feature vectors and cluster them into two sets (first lines and all the rest). We make use of a simple observation that the first line from all references blocks is also the first line of its reference. Thus, the cluster containing this first line is assumed to contain all first lines. After recognizing all first lines, it is easy to concatenate lines to form consecutive reference strings.

For clustering lines, we use KMeans algorithm with Euclidean distance metric. In this case \(K = 2\), since the line set is clustered into two subsets. As initial centroids, we set the first line’s feature vector and the vector with the largest distance to the first one. We use five features based on line relative length, line indentation, space between the line and the previous one, and the text content of the line (if the line starts with an enumeration pattern, if the previous line ends with a dot).

Reference strings extracted from references zones contain important reference metadata. In this step, metadata is extracted from reference strings and the result is the list of document’s parsed bibliographic references. The information we extract from the strings include: author, title, source, volume, issue, pages and year. An example of a parsed reference is shown in Fig. 8.

First a reference string is tokenized. The tokens are then transformed into vectors of features and classified by a supervised classifier. Finally, the neighbouring tokens with the same label are concatenated, the labels are mapped into final metadata classes and the resulting reference metadata record is formed.

The heart of the implementation is a classifier that assigns labels to reference tokens. For better performance, the classifier uses slightly more detailed labels than the target ones: first_name (author’s first name or initial), surname (author’s surname), title, source (journal or conference name), volume, issue, page_first (the lower bound of pages range), page_last (the upper bound of pages range), year and text (for separators and other tokens without a specific label). The token classifier employs conditional random fields and is built on top of GRMM and MALLET packages [35].

CRF classifiers are a state-of-the-art technique for citation parsing. They achieve very good results for classifying instances that form a sequence, especially when the label of one instance depends on the labels of previous instances.

The basic features are the tokens themselves. We use 42 additional features to describe the tokens:It is worth to notice that the token’s label depends not only on its feature vector, but also on the features of the surrounding tokens. To reflect this in the classifier, the token’s feature vector contains not only features of the token itself, but also features of two preceding and two following tokens.

After token classification, fragments labelled as first_name and surname are joined together based on their order to form consecutive author names, and similarly fragments labelled as page_first and page_last are joined together to form pages range. Additionally, in the case of title or source labels, the neighbouring tokens with the same label are concatenated.

The result of bibliography extraction is a list of document’s bibliographic references in a structured form, each of which contains the raw text as well as additional metadata.