A combined static and dynamic feature extraction technique to recognize handwritten digits

The recognition of scanned handwritten text is referred as offline handwriting recognition and the recognition of handwriting while writing is called online handwriting recognition. The present study has been experimented for scanned handwritten digit images. The recent advances in recognition of handwritten digits proves importance of this research topic and explain that alone recognition rate is not only important but also the stability and smoothness of developed system [2, 25]. The present study is an effort to recognize digit images using their static and dynamic properties. The one example of static property is the number of non-zero (white) pixels in binary images. The dynamic property includes features represent dynamic information and we have extracted these features from offline images after applying technique as recovery of drawing order. Most of the literature is available for recognition of scanned handwritten digits using static properties. Some of the selected literatures in context of handwritten digits using static properties are discussed here. A recent study includes novel prototype generation technique to recognize handwritten digits in two stages. An adaptive resonance theory-based algorithm used in first stage for an initial solution and second stage solve optimization problem associated to classification [7]. A hybrid convolution neural network and support vector machine model that automatically retrieves features based on convolution neural network is a recent study to recognize scanned digits [25]. In this work, convolution neural networks works as a feature extractor and support vector machine as a recognizer. This way the designed hybrid model extracts features from raw images and performs classification. A study related to recognition of digit images include multicolumn deep convolution neural network that able to extract features from an image [2]. Similar convolution neural network used to extract features and train network with different number of input and hidden layer neurons [16]. A deep neural net and hidden markov model-based recognizer developed where deep convolution network learning is batch-mode based. The deep convolution network observed to be an impressive technique in context of cpu computation time and classification [4]. An adaboost learning algorithm used to recognize digit images achieved significant error rate less than 1 % [9]. A feature extractor was applied to digit images where number of coefficient of images is computed on learned basis and a local maximum operation is performed [10]. This includes support vector machine-based training with proposed feature vector and designed system was compared against techniques as sparse coding, gabor wavelets and principal component analysis.

In present study of features from dynamic properties, these features are extracted after recovery of drawing order of digits in first step and computation of its chain codes in second step. The selected literature for recovery of drawing order has been discussed here. An instance-based stroke recovery from multistroke characters after selecting sufficient number of instances to cover various characters [8]. A recovery scheme of drawing order of handwritten letters after generating several paths and the best path is selected [23]. A survey to recover pen trajectories multiple approaches to recover drawing order as skeleton, contours, ambiguous zones detections and ambiguous zones analysis are discussed [17]. A study based on matching cost function that is the summation of positional distortions cost and directional difference cost between the template stroke and its matching path. A bidirectional search algorithm based on dynamic programming has been used to find best path [22]. A three phase approach used to restore writing order where first phase include neural network to obtain possible edge continuity relations, second phase include double traced lines identification and last third phase find candidates of single stroked paths by depth first search [21]. In other study, a method proposed to extract strokes from handwritten characters and graphemes. This method includes three stages as: vectorisation, merging of skeletal branches with loop recovery and adjustment of near junctions [19]. A recent study includes recovery of drawing order of handwritten digits where chain code directions have been used to traverse pixels of a digit image [24]. In this paper, an iterative method has been presented to traverse all pixels of an image where a pixel is visited one time and directions of trajectory is based on chain code computations. The chain code is a linear structure that results from quantization of the centers of adjacent boundary elements in an image array [6, 15]. The combined feature set of offline and online features have been used with recognition technique as support vector machine. The support vector machine is one of the popular classifier and has been used for recognition of digit images [3, 12].

The present study main objective is to use combined feature vector using static and dynamic features for a stable system. This paper includes six sections including this section. The system overview is presented in section two. The section three discusses feature computation of preprocessed images and the support vector machine for two or more classes recognition problem have been discussed in Sect. 4. The Sect. 5 presents outcome of experiments and last Sect. 6 conclude this paper with findings.

The present system development include major components as preprocessing, feature extraction, SVM training and recognition. The developed model include two stages as stage I and stage II. These stages are:

stage I: train set \(\rightarrow \) preprocessing \(\rightarrow \) feature extraction \(\rightarrow \) SVM training \(\rightarrow \) trained model.

stage II: test set \(\rightarrow \) preprocessing \(\rightarrow \) feature extraction \(\rightarrow \) Recognition using trained model \(\rightarrow \) output classes of test set.

The stage I focus on classifier training using train set and stage II recognizes the test set. In stage I, train set images are preprocessed first where images are resized and transformed to binary form. The features are computed and each image results in a feature vector. The train feature set is now trained with SVM and results in trained model as output of stage I for all classes in train set. The stage II preprocess and compute features vectors of test set images as happened in stage I. The computed feature vectors of test set images are recognized with trained model obtained through stage I and results in identification of classes for each image of test set.

The preprocessing part of stages I and II are not challenging in view to see the complexities of two tasks as it includes size normalization and binarization of images. The feature extraction technique results in feature vector of reasonable small length that makes our system lighter and help in fast execution as compared to the case where feature extraction is not applied. The details of preprocessing and feature extraction including suitable examples are discussed in next section. The SVM training is the step which needs maximum time for execution among all components of system as it trains large benchmarked data set mnist with 60,000 images. Our feature extraction scheme reduces length of feature vector to less than half of input feature vector length and save training time. The SVM training is batch mode in nature and small length of feature vector allow space to large amount of train data. The recognition using trained model in step II find all results for test image with respect to trained classes and the class with maximum number of counts is the recognized class. There may be instances where more than one class could have same number of counts; we select class with minimum index as the recognized class.

As discussed in Sect. 1, we have computed two types of features, namely, static and dynamic features. This section presents the procedure of preprocessing of image and their feature computations. The preprocessing of an image and their computation of static and dynamic features are explained in respective Sects. 3.1, 3.2 and 3.3.

The feature vector computed in previous section becomes input to recognition phase. The recognizer used in our work is SVM [26]. The SVM is one of the useful methods to classify linearly and non-linearly separable data. The technique of computation of support vectors has been explained in this section. For a given training data: The SVM solves labeled training data \((x_{i},y_{i}), i=1,2,3,\ldots n\) for the decision function \(h(x)=sign(w.z+b)\), where \(z=\phi _{z}(x) \in Z\) is a feature map of \(x\) [26]. The \(w\) and \(b\) are the solutions of: \(C\) is penalty parameter and \(\xi _{i}\) is slack variable. The SVM training is a large quadratic programming problem with computational complexity grows to \(O(n^{3})\). The computational complexity of its dual form is less as compared to primal formulation. The dual form of (1) derived through lagranges multipliers reduce the complexity of problem and dual form becomes:Here, \(\alpha _{i}\) is positive langrange multiplier. The decision function for dual form is: \(h(x)=\mathrm{sign}(\sum _{i=1}^N \alpha _{i} y_{i} z_{i}.z+b)\).

To classify non-linear separable data, a kernel trick is applied that takes data to classify in higher dimensional feature space \(\phi _{z}(x)\). The decision function including kernel becomes: \(h(x)=\mathrm{sign}(\sum _{i=1}^N \alpha _{i} y_{i} K(z_{i}.z)+b)\) and \(K(z_{i}.z_{j})=\phi _{z}(x_{i}).\phi _{z}(x_{j})\) is a valid kernel function that satisfy mercer condition. Some of the popular kernels are: (1) Linear, \(K(z_{i}.z_{j})=z_{i}.z_{j}\), (2) Gaussian, \(K(z_{i}.z_{j})=exp(-\gamma \Vert z_{i}-z_{j}\Vert ^2)\), (3) Polynomial, \(K(z_{i}.z_{j})=(p+z_{i}.z_{j})^q\), (4) Sigmoid, \(K(z_{i}.z_{j})~=~\mathrm{tan}h(k.z_{i}.z_{j}-\delta )\).

A large quadratic programming problem breaks into small quadratic problems with two variables at a time in sequential minimal optimization (SMO) algorithm proposed by [20]. The SMO algorithm common stages as: computation of working set of two variables and updating respective lagranges multipliers of current working set variables. These stages are repeatedly used until convergence level is achieved. We have used second order information to compute working set as proposed in literature by [5]. This second order information helps in faster execution of SMO algorithm and achieves minimum number of working sets. These working set pairs are referred as maximal violating pair where violation is subject to KKT conditions [1]. For a maximal violating pair \(B(i,j)\):This allows working set to checks only \(O(n)\) possible pairs for selecting \(j\). Let \(a_{ij}=(z_{i},z_{i})+(z_{j},z_{j})-2(z_{i},z_{j})\) and \(b_{ij}=-y_{i}\nabla f(\alpha ^{k})_{i}+y_{j}\nabla f(\alpha ^{k})_{j}\), this makes working set selection as:The updated lagranges multiplier for \(i,j\) is as follows:The \(\tau \) is a constant. Then, \(\alpha _{i}^\mathrm{new}\) and \(\alpha _{j}^\mathrm{new}\) need to be clipped in range [0,C] as discussed in SMO algorithm.before \(\alpha _{j}^\mathrm{new,clipped}\), the updated \(\alpha _{i}^\mathrm{new}\) need to use for \(\alpha _{j}^\mathrm{new}\) as: The updated lagranges multipliers are multiplied with respective targets as ‘\(-\)1’ and ‘+1’ to either side of classification boundaries and final values of \(\alpha \)’s are referred as support vectors. To solve multiclass problem, one-against-one approach has been opted. In this approach for \(m\) class problem, it constructs \(m(m-1)/2\) classifiers, where each classifier uses the training data from two classes chosen out of \(m\) classes. The present experiments uses ‘max win’ approach where a class is recognized that wins maximum votes. In any case, classes with same number of votes, the class with smallest index is accepted.

The computed feature vector has been used with SVM recognition technique and experiments are conducted using benchmark dataset mnist [13]. The mnist dataset include 60,000 training images and 10,000 test images. The experiments are conducted using scientific tools as matlab and libsvm [5]. Each image of the train and test set transformed to feature vector of length 356. The transformed train set undergoes SVM training and results in trained set with computed support vectors for individual classes. This trained set is used to recognize classes of transformed test set. The polynomial kernel used in SVM recognition method achieved better results as compared to other kernels as linear, rbf and sigmoid. The Table 2 presents results with polynomial kernel with degrees 3, 5, 7, 9, 11 and 13. The results show that degree 7 and 9 achieve better results as compared to degrees 3, 5, 11 and 13. The best recognition accuracy result achieved as 99.27 % (0.73 % as error rate) using polynomial degree 7. The column average recognition per class in Table 2 presents recognition accuracy for classes from 0 to 9 using kernel degrees 3, 5, 7, 9, 11 and 13. The average result for each class shows that the present system capability under different kernel degrees to behave stable. In literature, we have observed very low error rate for used data set mnist as 0.19 % [25]. Our prime focus is not to achieve lowest error rate for used data set mnist but to propose a technique with hybrid features where static and dynamic features could be used together. The proposed feature extraction technique is challenging in nature as it includes drawing order of an image where original drawing order is not known. Our reported error rate as 0.73 % give scope for future work to extend present study as reported error rate justify that proposed technique is promising. The confusion matrix for reported error rate as 0.73 % has been presented in Table 3 where class ‘9’ shows maximum number of incorrect predictions as ‘14’ and class ‘0’ shows minimum incorrect prediction as ‘3’. The images for result of confusion matrix have been presented in Fig. 12 with labels \(x \rightarrow y\) for each result in image. The \(x\) presents true label and \(y\) presents predicted label. The incorrect recognized images vision to human recognition has been explained for mnist data set in literature [12]. We also find some of our images same for which incorrect recognition happened in literature.

As present study includes implementation of hybrid features comprising static and dynamic in nature, we have separately recognized mnist data set for static and dynamic features. These separate test results help in understanding of effectiveness of static and dynamic features exclusively. The Table 4 present results for static and dynamic features as 1.46 and 1.78 % error rates, respectively. These two error rates are higher as compare to result reported using static and dynamic features together. The error rate for handwritten digit recognition with mnist data set has been reported below one percent in recent study as referred in literature. We find that our scheme do not achieve minimum error rate as compared to literature but we are able to achieve error rate below 1 % that is at par with the work done in this area. Some of the selected previous results are presented in Table 5 where we find that our approach report error rate 0.73 % with feature vector length per image as 356. In Table 5, we have presented selective results of literature where mnist data set have been used. The mnist data set have been used extensively in literature. Therefore, it is not easy to present all results from literature. The progress made in recognition of mnist data set using different approaches is available in [14]. Our results indicate that proposed hybrid feature technique results in smaller feature vector length as 356. The feature vector length as 356 is small as compared to 784 which has been commonly used in literature as presented in Table 5. The selection of small feature vector length makes our system lighter and capable to work better with machines having memory constraints. Also, our approach include technique of feature extraction of an image that presents uniqueness and novelty of our approach to understands images through both static and dynamic behavior.

As discussed in section one that objective of present study to provide a novel feature scheme that merges both static and dynamic handwriting features. We observed that our system is stable from study observed in experiments section and able to achieve error rate less than 1 % which is at par with the work done in this area. We have observed following important outcomes of study:The work with combined feature scheme is new and need to be implemented for more complex systems where recovery of drawing order is a challenging task. The recovery of drawing order becomes more challenging in the cases where original drawing order is not known. The recovery of drawing order has attracted researchers in recent past as it has important applications in the field of biometric systems. The applications of this field could be applied to financial systems, signature verifications and documents recognition where static and dynamic information can be used together.