Elsevier

Information Fusion

Volume 18, July 2014, Pages 78-85
Information Fusion

License plate detection based on multistage information fusion

https://doi.org/10.1016/j.inffus.2013.05.008Get rights and content

Abstract

Adaboost detector has been successfully used in object detection. In this paper, we propose a new License Plate (LP) detection technique based on multistage information fusion, which is adopted to reduce high false alarm rate in the conventional Adaboost detector. The proposed multistage information fusion system is composed of an enhanced Adaboost detector, a color checking module and an SVM detector, where the latter two stages further check whether the image patch that gets through the Adaboost detector is an LP. Test results of the dataset that consists of 950 real-world images show that the fusion reduces the false alarm rate. The proposed Fusion detector outperforms the conventional Adaboost detector throughout the ROC (Receiver Operating Characteristic) curve. The AUC (Area Under the Curve) of the best Fusion detector reaches 0.9081; however, the AUC of the best Adaboost detector is only 0.8441, which shows that the modification on feature extraction and the multistage information fusion significantly improve the LP detection performance.

Introduction

A license plate recognition (LPR) system is an important part of intelligent transportation systems (ITSs). It is used in a wide range of applications including parking charges, pay-per-use road usage and traffic law enforcement, etc. In general, a license plate recognition system is divided into three steps: (1) detecting the license plate; (2) extracting the characters from the license plate one by one; and (3) recognizing the characters from (2). Among the above three steps, License Plate (LP) detection is the most difficult step. If an LP is precisely detected, further processing becomes rather simple. In a practical context, LP detection in an inconstant environment is difficult. The variable factors include illumination variance, complex background, unpredictable weather and tilts caused by viewing angle. These problems are particularly intolerable when the system is used in an outdoor environment.

In the past few decades, LP detection has generated much research. There are three major categories of methods for LP detection: The first category is statistical or morphological operations of low level visual features, such as color, edge, etc. [1], [2], [3], [4], [5]. This is the most frequently used LP detection method in the early LPR systems. Visual attention model presents an appropriate framework to integrate the low level features [6]. The second category utilizes middle level features for LP detection, in which shape [7] and symmetry [8] are taken into consideration. Texture analysis by Gabor filter [9], wavelet transform [10] and Hough transform [11] also give impressive results in LP detection. The last category is the methods based on artificial intelligence (AI), including neural networks (NN) [12], [13], Adaboost [14], etc. Support vector machine (SVM) is a powerful texture classifier [15]; however, due to its heavy computation requirements, SVM has not been used in LP detection directly. In reference [16], the authors first estimated LP region by motion information, then verified the result by SVM. In an AI based LP detection method, the classifier scans the whole image patch by patch, and determines whether it is an LP. Most of these works perform well in simple and constant backgrounds; but they suffer performance degradation if the background is complex or the environment (such as illumination, climate, viewing angle, distance) changes. Simply put, the challenges of LP detection are still far from being solved.

Since Adaboost has had great success in face detection [17], Dlagnekov tried to apply it to detect license plates and achieved a detection rate of 95.6% with a false alarm rate of 5.7% [14]. Pan et al. tried to detect Chinese license plates using Adaboost [18], whose test results on 200 frames yielded a detection rate of 81% with a false alarm rate of 6.5%. Their results are with a high false alarm rate, not as good as expected. From all the above we can conclude that, the Adaboost LP detectors are far from practical use.

This paper presents a multistage information fusion detector for LP detection, in which information fusion is applied to improve the conventional Adaboost LP detector. The direct contributions of this work are improved feature extraction and multistage information fusion of detectors. The feature extraction of [14], [18] for LP detection does not look into the details of LPs. In our implementation, the Harr-like features are restricted in the scale of characters and strokes, which are more meaningful for LP detection. Besides modifications and restraints on feature extraction, a multistage information fusion detector, which is composed of an enhanced Adaboost detector, a color checking module and an SVM detector, reduces the false alarm rate effectively. Tests on 950 real-world car images in complex background show encouraging results, which proves that the modifications on feature extraction and the multistage information fusion do improve the performance of LP detection.

The rest of the paper is organized as follows: Section 2 describes details of a license plate detection algorithm based on Adaboost. In Section 3, a multistage information fusion procedure composed of an Adaboost detector, a color checking module and an SVM detector is presented. Test results on a dataset consisting of real-world car images are shown and discussed in Section 4. Finally, we conclude this paper and look forward to future work in Section 5.

Section snippets

Adaboost detector – training and detection

This section provides a detailed description of Adaboost detector. Section 2.1 presents the Harr-like feature extraction for LP detection. Section 2.2 establishes a method of training the Adaboost LP detector offline. And Section 2.3 outlines the online work procedure of the Adaboost LP detector.

LP checking by multistage information fusion

As shown by results in [14], [18], the conventional Adaboost detector in real LP detection application achieves good performance; however, there is still room to reduce the high false alarm rate. It is reasonable to presume that additional checks would be helpful to reduce the false alarms, and it is found to be the case for the application shown in this work.

Color checking [4], [21] has been applied to LP detection, and have achieved encouraging performance. In Ref. [16], the SVM algorithm is

Experimental results

In this section, the detection method is verified by a dataset composed of real-world car images. A surveillance camera was installed at the left side of an exit of our campus. 950 images were captured during 11 h with constantly changing context. The resolution of each image is 576 * 720, and 858 of the 950 images in the dataset include a car with a license plate. The other 92 images in the dataset contain no license plate, among which some include very small sections of LP, some include a new

Conclusions

Conventional Adaboost detector is enhanced by multistage information fusion for LP detection. Major modifications include: First, the Harr-like feature extraction is modified and restricted in the scale of character and stroke to accommodate for LP detection. Second, multistage information fusion is adopted to reduce false alarm rate of the conventional Adaboost LP detector. The whole multistage information fusion system is composed of an enhanced Adaboost detector, a color checking module and

Acknowledgments

The authors thank the anonymous reviewers, their comments and suggestions helped to improve the quality of the paper. We also want to thank Mingye Wang for his patient language editing. This work was supported by grants partly from the National Science and Technology Major Project (No. 2010ZX03006-001-02), and partly from One Hundred Talents Project of The Chinese Academy of Sciences (No. 99M2008M02).

References (24)

  • D. Zheng et al.

    An efficient method of license plate location

    Pattern Recognition Letters

    (2005)
  • C. Anagnostopoulos et al.

    License plate recognition from still images and video sequences: a survey

    IEEE Transactions on Intelligent Transportation Systems

    (2008)
  • S. Wang et al.

    Detection and recognition of license plate characters with different appearances

    2003 IEEE Proceedings on Intelligent Transportation Systems

    (2003)
  • X. Shi et al.

    Automatic license plate recognition system based on color image processing

    Lecture Notes on Computer Science

    (2005)
  • B. Hongliang, L. Changping, A hybrid license plate extraction method based on edge statistics and morphology, in:...
  • Z. Yao, W. Yi, License plate location based on improved visual attention model, in: International Conference on Machine...
  • C. Oz et al.

    A practical license plate recognition system for real-time environments

  • D. Kim, S. Chien, Automatic car license plate extraction using modified generalized symmetry transform and image...
  • F. Kahraman et al.

    License plate character segmentation based on the Gabor transform and vector quantization

    Lecture Notes in Computer Science

    (2003)
  • C. Hsieh, Y. Juan, K. Hung, Multiple license plate detection for complex background, in: Proceedings of the 19th...
  • C. Nguyen, M. Ardabilian, L. Chen, Real-time license plate localization based on a new scale and rotation invariant...
  • M. Chacon, A. Zimmerman, License plate location based on a dynamic PCNN scheme, in: Proc. Int. Joint Conf. Neural...
  • Cited by (26)

    • Data fusion for ITS: A systematic literature review

      2023, Information Fusion
      Citation Excerpt :

      The model fused different heterogeneous inputs from Taxi’s GPS, historical traffic accident data, POI distributions, and weather observations at various road networks. Vehicle type detection: Yao et al. [197] adopt a multi-stage information fusion to improve the performance of the conventional Adaboost vehicle’s license plate detector. Li et al. [163]adopt a multistage information fusion to enhance the performance of the traditional Adaboost vehicle’s license plate detector.

    • Multiwavelet transform based license plate detection

      2017, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      Every country has its own License Plate Recognition (LPR) system because LP is country specific. ANPR systems installed at many survey points are used to collect information of the vehicle for automatic toll collection, car parking, pay-per-use road usage, traffic law enforcement [2,3] etc. It is mature yet imperfect field because of different styles of LP, region-wise format, small LP region with different font size, inclined LP, different color variation and various outdoor conditions.

    • A new data classification method based on chaotic particle swarm optimization and least square-support vector machine

      2015, Chemometrics and Intelligent Laboratory Systems
      Citation Excerpt :

      Zhang [17] proposed an improved data classification method based on SVM applying rational sample data selection and GA-controlled training parameters optimization. Yao and Yi [18] proposed a new License Plate (LP) detection technique based on multistage information fusion. Sung and Chung [19] proposed a distributed energy monitoring network system based on data fusion via improved PSO algorithm.

    View all citing articles on Scopus
    View full text