Skip to main content
SpringerLink
Log in

Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Fruit category identification is important in factories, supermarkets, and other fields. Current computer vision systems used handcrafted features, and did not get good results. In this study, our team designed a 13-layer convolutional neural network (CNN). Three types of data augmentation method was used: image rotation, Gamma correction, and noise injection. We also compared max pooling with average pooling. The stochastic gradient descent with momentum was used to train the CNN with minibatch size of 128. The overall accuracy of our method is 94.94%, at least 5 percentage points higher than state-of-the-art approaches. We validated this 13-layer is the optimal structure. The GPU can achieve a 177× acceleration on training data, and a 175× acceleration on test data. We observed using data augmentation can increase the overall accuracy. Our method is effective in image-based fruit classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Acquarelli J, van Laarhoven T, Gerretzen J et al (2017) Convolutional neural networks for vibrational spectroscopic data analysis. Anal Chim Acta 954:22–31

    Article  Google Scholar 

  2. Adak MF, Yumusak N (2016) Classification of E-Nose Aroma Data of Four Fruit Types by ABC-Based Neural Network. Sensors 16(3):13

    Article  Google Scholar 

  3. Ahmad J, Mehmood I, Baik SW (2017) Efficient object-based surveillance image search using spatial pooling of convolutional features. J Vis Commun Image Represent 45:62–76

    Article  Google Scholar 

  4. Bai X, Shi BG, Zhang CQ et al (2017) Text/non-text image classification in the wild with convolutional neural networks. Pattern Recogn 66:437–446

    Article  Google Scholar 

  5. Chen Y (2016) Voxelwise detection of cerebral microbleed in CADASIL patients by leaky rectified linear unit and early stopping: a class-imbalanced susceptibility-weighted imaging data study. Multime Tools Appl. https://doi.org/10.1007/s11042-017-4383-9

  6. Cicero M, Bilbily A, Dowdell T et al (2017) Training and Validating a Deep Convolutional Neural Network for Computer-Aided Detection and Classification of Abnormalities on Frontal Chest Radiographs. Investig Radiol 52(5):281–287

    Article  Google Scholar 

  7. Cintas C, Quinto-Sanchez M, Acuna V et al (2017) Automatic ear detection and feature extraction using Geometric Morphometrics and convolutional neural networks. IET Biometrics 6(3):211–223

    Article  Google Scholar 

  8. Dai-Ton H, Duc-Dung N, Duc-Hieu L (2016) An adaptive over-split and merge algorithm for page segmentation. Pattern Recogn Lett 80:137–143

    Article  Google Scholar 

  9. Deliens T, Deforche B, Annemans L et al (2016) Effectiveness of pricing strategies on french fries and fruit purchases among university students: results from an on-campus restaurant experiment. PLoS One 11(11):16 Article ID: e0165298

    Article  Google Scholar 

  10. Di Cagno R, Filannino P, Cavoski I et al (2017) Bioprocessing technology to exploit organic palm date (Phoenix dactylifera L. cultivar Siwi) fruit as a functional dietary supplement. J Funct Foods 31:9–19

    Article  Google Scholar 

  11. Garcia F, Cervantes J, Lopez A et al (2016) Fruit classification by extracting color chromaticity, shape and texture features: towards an application for supermarkets. IEEE Lat Am Trans 14(7):3434–3443

    Article  Google Scholar 

  12. Getahun S, Ambaw A, Delele M et al (2017) Analysis of airflow and heat transfer inside fruit packed refrigerated shipping container: Part I - model development and validation. J Food Eng 203:58–68

    Article  Google Scholar 

  13. Ghazi MM, Yanikoglu B, Aptoula E (2017) Plant identification using deep neural networks via optimization of transfer learning parameters. Neurocomputing 235:228–235

    Article  Google Scholar 

  14. Ji G (2014) Fruit classification using computer vision and feedforward neural network. J Food Eng 143:167–177

    Article  Google Scholar 

  15. Jiang YL, Zur RM, Pesce LL et al (2009) A Study of the Effect of Noise Injection on the Training of Artificial Neural Networks. In International Joint Conference on Neural Networks (IJCNN), IEEE, Atlanta, pp 2784–2788

  16. Kim JH, Hong HG, Park KR (2017) Convolutional neural network-based human detection in nighttime images using visible light camera sensors. Sensors (Basel) 17(5). https://doi.org/10.3390/s17051065

  17. Kooi T, van Ginneken B, Karssemeijer N et al (2017) Discriminating solitary cysts from soft tissue lesions in mammography using a pretrained deep convolutional neural network. Med Phys 44(3):1017–1027

    Article  Google Scholar 

  18. Lee CH, Chien JT (2016) Deep unfolding inference for supervised topic model. In International Conference on Acoustics, Speech And Signal Processing Proceedings, IEEE, Shanghai, pp 2279–2283

  19. Li S, Jiang H, Pang W (2017) Joint multiple fully connected convolutional neural network with extreme learning machine for hepatocellular carcinoma nuclei grading. Comput Biol Med 84:156–167

    Article  Google Scholar 

  20. Liu F, Snetkov L, Lima D (2017) Summary on fruit identification methods: A literature review. Adv Soc Sci Educ Hum Res 119:1629–1633

    Google Scholar 

  21. Lu Z (2016) Fractional Fourier entropy increases the recognition rate of fruit type detection. BMC Plant Biol 16(S2) Article ID: 10

  22. Lu Z, Li Y (2017) A fruit sensing and classification system by fractional fourier entropy and improved hybrid genetic algorithm. In 5th International Conference on Industrial Application Engineering (IIAE). Kitakyushu, Institute of Industrial Applications Engineers, Japan, pp 293–299

  23. Miki Y, Muramatsu C, Hayashi T et al (2017) Classification of teeth in cone-beam CT using deep convolutional neural network. Comput Biol Med 80:24–29

    Article  Google Scholar 

  24. Oliva D, Hinojosa S, Cuevas E et al (2017) Cross entropy based thresholding for magnetic resonance brain images using Crow Search Algorithm. Expert Syst Appl 79:164–180

    Article  Google Scholar 

  25. Pardo-Mates N, Vera A, Barbosa S et al (2017) Characterization, classification and authentication of fruit-based extracts by means of HPLC-UV chromatographic fingerprints, polyphenolic profiles and chemometric methods. Food Chem 221:29–38

    Article  Google Scholar 

  26. Qian RQ, Yue Y, Coenen F et al (2016) Traffic sign recognition with convolutional neural network based on max pooling positions. In 2th International Conference on Natural Computation, Fuzzy Systems And Knowledge Discovery (ICNC-FSKD), IEEE, Changsha pp 578–582

  27. Radi, Ciptohadijoyo S, Litananda WS et al (2016) Electronic nose based on partition column integrated with gas sensor for fruit identification and classification. Comput Electron Agric 121:429–435

    Article  Google Scholar 

  28. Shao WH, Li YJ, Diao SF et al (2017) Rapid classification of Chinese quince (Chaenomeles speciosa Nakai) fruit provenance by near-infrared spectroscopy and multivariate calibration. Anal Bioanal Chem 409(1):115–120

    Article  Google Scholar 

  29. Sui XD, Zheng YJ, Wei BZ et al (2017) Choroid segmentation from Optical Coherence Tomography with graph edge weights learned from deep convolutional neural networks. Neurocomputing 237:332–341

    Article  Google Scholar 

  30. Tabik S, Peralta D, Herrera-Poyatos A et al (2017) A snapshot of image pre-processing for convolutional neural networks: case study of MNIST. Int J Comput Intellig Syst 10(1):555–568

    Article  Google Scholar 

  31. Teh V, Sim KS, Wong EK (2016) Brain early infarct detection using gamma correction extreme-level eliminating with weighting distribution. Scanning 38(6):842–856

    Article  Google Scholar 

  32. Thung KH, Paramesran R, Lim CL (2012) Content-based image quality metric using similarity measure of moment vectors. Pattern Recogn 45(6):2193–2204

    Article  Google Scholar 

  33. Thung KH, Wee CY, Yap PT et al (2014) Neurodegenerative disease diagnosis using incomplete multi-modality data via matrix shrinkage and completion. NeuroImage 91:386–400

    Article  Google Scholar 

  34. Thung KH, Wee CY, Yap PT et al (2016) Identification of progressive mild cognitive impairment patients using incomplete longitudinal MRI scans. Brain Struct Funct 221(8):3979–3995

    Article  Google Scholar 

  35. Tovar MF, Losada HV (2016) Fuzzy systems: case study classification of fruit Mc Stipitata Vaug (Araza). Amazonia Investiga 5(9):45–56

    Google Scholar 

  36. Wei L (2015) Fruit classification by wavelet-entropy and feedforward neural network trained by fitness-scaled chaotic ABC and biogeography-based optimization. Entropy 17(8):5711–5728

    Google Scholar 

  37. Wu L (2012) Classification of fruits using computer vision and a multiclass support vector machine. Sensors 12(9):12489–12505

    Article  Google Scholar 

  38. Wu J (2016) Fruit classification by biogeography-based optimization and feedforward neural network. Expert Syst 33(3):239–253

    Article  Google Scholar 

  39. Smirnov EA, Timoshenko DM, Andrianov SN (2014) Comparison of regularization methods for imagenet classification with deep convolutional neural networks. In 2nd Aasri Conference on Computational Intelligence And Bioinformatics (CIB). Elsevier Science Bv, South Korea, pp 89–94

  40. Yaghoubi S, Noori S, Azaron A et al (2015) Resource allocation in multi-class dynamic PERT networks with finite capacity. Eur J Oper Res 247(3):879–894

    Article  MathSciNet  Google Scholar 

  41. Zhang Y (2016) GroRec: a group-centric intelligent recommender system integrating social, mobile and big data technologies. IEEE Trans Serv Comput 9(5):786–795

    Article  Google Scholar 

  42. Zhang Y, Qiu M, Tsai CW et al (2015) Health-CPS: healthcare cyber-physical system assisted by cloud and big data. IEEE Syst J PP(99):1–8

    Google Scholar 

  43. Zhang Y, Chen M, Huang D et al (2017) iDoctor: Personalized and professionalized medical recommendations based on hybrid matrix factorization. Futur Gener Comput Syst 66:30–35

    Article  Google Scholar 

  44. Zhu SG, Du JP (2014) Visual tracking using max-average pooling and weight-selection strategy. J Appl Math 2014:828907

    Google Scholar 

Download references

Acknowledgments

This study was supported by Natural Science Foundation of China (61602250), Natural Science Foundation of Jiangsu Province (BK20150983), Open fund of Key Laboratory of Guangxi High Schools Complex System and Computational Intelligence (2016CSCI01).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, Henan, 454000, People’s Republic of China

    Yu-Dong Zhang & Shui-Hua Wang

  2. Jiangsu Key Laboratory of Advanced Manufacturing Technology, Huaiyin, Jiangsu, 223003, China

    Yu-Dong Zhang

  3. Translational Imaging Division & MRI Unit, Columbia University and New York State Psychiatric Institute, New York, NY, 10032, USA

    Zhengchao Dong

  4. Department of electrical engineering, College of engineering, Zhejiang Normal University, Jinhua, Zhejiang, 321004, China

    Xianqing Chen

  5. School of Computer Science and Technology, Nanjing Normal University, Nanjing, Jiangsu, 210023, China

    Wenjuan Jia

  6. School of Electronic Science and Engineering, Nanjing University, Nanjing, Jiangsu, 210046, China

    Sidan Du

  7. College of Software Convergence, Sejong University, Seoul, Republic of Korea

    Khan Muhammad

Authors
  1. Yu-Dong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengchao Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianqing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenjuan Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sidan Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Khan Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shui-Hua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yu-Dong Zhang, Khan Muhammad or Shui-Hua Wang.

Ethics declarations

Conflict of interest

We have no conflicts of interest to disclose with regard to the subject matter of this paper.

Additional information

Highlights

• We proposed a 13-layer convolutional neural network, and validated the optimal number of convolution layers and pooling layers.

• We validated that the max pooling gives better slight performance than average pooling.

• Our method yielded an overall accuracy of 94.94%, better than five state-of-the-art approaches.

• We tested our method on imperfect images. The overall accuracy over background fruit images is 89.60%, over decay images is 94.12%, over unfocused images is 91.03%, and over occlusion image is 92.55%.

• We compared CPU and GPU computation, and found GPU can achieve a 177× acceleration on training data, and a 175× acceleration on test data.

• We used five different types of data augmentation methods, and compared the classification performance of using data augmentation and not using data augmentation.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, YD., Dong, Z., Chen, X. et al. Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation. Multimed Tools Appl 78, 3613–3632 (2019). https://doi.org/10.1007/s11042-017-5243-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5243-3

Keywords

Search

Navigation