Skip to main content
SpringerLink
Log in

High-efficiency face detection and tracking method for numerous pedestrians through face candidate generation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper is dedicated to developing high-efficiency face detection and tracking method for big dynamic crowds or numerous pedestrians. Three modules constitute the proposed method, i.e., face candidate generation, face candidate verification, and face target tracking. In this work, face candidates are localized using the features of the face area, edge information, and skin color. Non-face parts in the face candidates are further verified by the C-SVM learning model and then removed, by which the face targets can be generated with lower computation-complexity and satisfactory accuracy than other approaches. Finally, the face targets are tracked by an efficient and reliable searching scheme for improving the effective face detection rate. Experimental results show that the average face detection rate (FDR) of 85%, average effective FDR of 95%, a frame rate of 28–66 frames per second (fps), and about 30 faces detected per frame are obtained from various test videos with big dynamic crowds or numerous pedestrians, indicating the feasibility of the proposed method to achieve unconstrained face detection with high-efficiency and cost-effectiveness. This result makes the proposed method more attractive for the video surveillance system as compared to other approaches, especially in the high computational complexity-based methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Bochkovskiy A, Wang C-Y, Liao HYM (2020) YOLOv4: Optimal speed and accuracy of object detection, arXiv preprint: arXiv:2004.10934v1, 23 Apr. 2020

  2. Cárdenas RJ, Beltrán CA, Gutiérrez JC (2019) Small face detection using deep learning on surveillance videos. Int J Mach Learn Comput 9(2):189–194

    Article  Google Scholar 

  3. Chen C-H, Chen T-Y, Huang D-Y, Hu W-C, Guo Z-B, Wen C-K (2018) Real-time face detection in big crowd through face candidates. In: Proc. of 8th international congress on engineering and information, pp. 76-77

  4. Chen J-C, Rajeev R, Swami S, Amit K, Chen C-H, Vishal M-P, Castillo D-C, Rama C (2018) Unconstrained still/video-based face verification with deep convolutional neural networks. Int J Comput Vis 126(4):272–291

    Article  MathSciNet  Google Scholar 

  5. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. Proc Int Conf Comput Vis Pattern Recog 1:886–893

    Google Scholar 

  6. Fukui H, Yamashita T, Yamauchi Y, Fujiyoshi H, Murase H (2015) Pedestrian detection based on deep convolutional neural network with ensemble inference network. IEEE Intell Veh Symp (IV) 223-228

  7. Hoiem D, Efros A-A, Hebert M (2008) Putting objects in perspective. Int J Comput Vis 80(1):3–15

    Article  Google Scholar 

  8. Hu W-C, Chen C-H, Chen T-Y, Huang D-Y, Wu Z-C (2015) Moving object detection and tracking from video captured by moving camera. J Vis Commun Image Represent 30:164–180

    Article  Google Scholar 

  9. Huang D-Y, Chen C-H, Chen T-Y, Hu W-C, Lin Y-L (2016) A vehicle flow counting system in rainy environment based on vehicle feature analysis. J Inf Hiding Multimed Signal Process 7(1):101–114

    Google Scholar 

  10. Ishii Y, Hongo H, Yamamoto K, Niwa Y (2004) Face and head detection for a real-time surveillance system. Proc Int Conf Pattern Recog 3:298–301

    Article  Google Scholar 

  11. Ishii I, Ichida T, Gu Q, Takaki T (2013) 500-fps face tracking system. J Real-Time Image Process 8(4):379–388

    Article  Google Scholar 

  12. Ji S, Lu X, Xu Q (2014) A fast face detection method combining skin color feature and adaboost. Proc Int Conf Multisensor Fusion Inf Integr Intell Syst 1–5

  13. Ji P, Kim Y, Yang Y, Kim Y-S (2016) Face occlusion detection using skin color ratio and LBP features for intelligent video surveillance systems. Proc Federated Conf Comput Sci Inf Sys 253–259

  14. Jin H, Liu Q, Lu H, Tong X (2004) Face detection using improved LBP under Bayesian framework. Proc Int Conf Image Graph 306–309

  15. Krizhevsky A, Sutskever I, Hinton G-E (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 1097–1105

  16. Kwon H-J, Lee S-H, Hosseini S, Moon J, Koo H-I, Cho N-I (2017) Multiple face tracking method in the wild using color histogram features. IEEE Int Symp Signal Process Inf Technol 51–55

  17. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  18. Li Z, Xue L, Tan F (2010) Face detection in complex background based on skin color features and improved AdaBoost algorithm. Proc IEEE Int Conf Prog Inf Comput 2:723–727

    Google Scholar 

  19. Li J, Karmoshi S, Zhu M (2017) Unconstrained face detection based on cascaded convolutional neural networks in surveillance video. Proc Int Conf Image Vis Comput 46–52

  20. Marciniak T, Chmielewska A, Weychan R, Parzych M, Dabrowski A (2015) Influence of low resolution of images on reliability of face detection and recognition. Multimed Tools Appl 74(12):4329–4349

    Article  Google Scholar 

  21. Mita T, Kaneko T, Hori O (2005) Joint Haar-like features for face detection. Proc Int Conf Comput Vis 2:1619–1626

    Google Scholar 

  22. Niu G, Chen Q (2018) Learning an video frame-based face detection system for security fields. J Vis Commun Image Represent 55:457–463

    Article  Google Scholar 

  23. Online available: https://data.gov.tw/dataset/40584. Accessed 3 August 2020

  24. Phung S-L, Bouzerdoum A, Chai D (2002) A novel skin color model in YCbCr color space and its application to human face detection. Proc Int Conf Image Process 1:289–292

    Article  Google Scholar 

  25. Qian Y, Bi M, Tan T, Yu K (2016) Very deep convolutional neural networks for noise robust speech recognition. IEEE/ACM Trans Audio Speech Lang Process 24(12):2263–2276

    Article  Google Scholar 

  26. Rajeshwari J, Karibasappa K, Gopal Krishna M-T (2014) Survey on skin based face detection on different illumination poses and occlusion. Proc Int Conf Contemp Comput Inf 728–733

  27. Redmon J, Farhadi A (2018) YOLOv3: An incremental improvement, arXiv preprint: arXiv:180402767v2, 8 Apr. 2018

  28. Rekha N, Kurian M-Z (2014) Face detection in real time based on HOG. Int J Adv Res Comput Eng Technol 3(4):1345–1352

    Google Scholar 

  29. Sobel I, Feldman G (1968) A 3×3 isotropic gradient operator for image processing. A talk at the Stanford Artificial Project 271–272

  30. Sobottka K, Pitas I (1998) A novel method for automatic face segmentation, facial feature extraction and tracking. Signal Process Image Commun 12(3):263–281

    Article  Google Scholar 

  31. Tan M, Le QV (2019) EfficientNet: rethinking model scaling for convolutional neural network, arXiv preprint: arXiv:190511946v3, 23 Nov. 2019

  32. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Proc Int Conf Comput Vis Pattern Recog 511–518

  33. Xiong Z, Yao Z, Ma Y, Wu X (2019) VikingDet: a real-time person and face detector for surveillance cameras. Proc IEEE Int Conf Adv Video Signal Based Surveill 1:1–7

    Google Scholar 

  34. Yu M, Yun L, Chen Z, Cheng F (2017) Research on video face detection based on AdaBoot algorithm training classifier. Proc Int Conf Electron Instrumen Inf Syst 1–6

  35. Zhang L, Chu R, Xiang S, Liao S, Li S-Z (2007) Face detection based on multi-block LBP representation. Proc Int Conf Biom 11–18

  36. Zulhadi Z, Shahrel A-S, Junita M-S (2018) Hierarchical skin-AdaBoost-neural network (H-SKANN) for multi-face detection. Appl Soft Comput 68(7):172–190

    Google Scholar 

Download references

Acknowledgements

This work was partly supported by a grant from Ministry of Science and Technology, Taiwan, under the contracts MOST 107-2221-E-212-012, MOST 107-2622-E-992-024-CC3 and MOST 106-2221-E-151-061.

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, Da-Yeh University, 168 University Rd., Dacun, Changhua, 515, Taiwan, Republic of China

    Deng-Yuan Huang

  2. Department of Electronic Engineering, National Kaohsiung University of Science and Technology, 415 Chien Kung Rd., Kaohsiung, 807, Taiwan, Republic of China

    Chao-Ho Chen, Tsong-Yi Chen & Zhi-Bin Guo

  3. Department of Computer Science and Information Engineering, National Penghu University of Science and Technology, 300 Liu-Ho Rd., Makung, Penghu, 880, Taiwan, Republic of China

    Wu-Chih Hu

  4. Department of Information Management, Tainan University of Technology, 529 Zhongzheng Rd., Yongkang District, Tainan, 71002, Taiwan, Republic of China

    Cheng-Kang Wen

Authors
  1. Deng-Yuan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao-Ho Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tsong-Yi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wu-Chih Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhi-Bin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cheng-Kang Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Ho Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, DY., Chen, CH., Chen, TY. et al. High-efficiency face detection and tracking method for numerous pedestrians through face candidate generation. Multimed Tools Appl 80, 1247–1272 (2021). https://doi.org/10.1007/s11042-020-09780-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09780-y

Keywords

Search

Navigation