Skip to main content
Erschienen in: Information Systems Frontiers 5/2020

25.06.2020

Evolutionary Programming Based Deep Learning Feature Selection and Network Construction for Visual Data Classification

Erschienen in: Information Systems Frontiers | Ausgabe 5/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Convolutional Neural Network (CNN) models and many accessible large-scale public visual datasets have brought lots of research work to a remarkable new stage. Benefited from well-trained CNN models, small training datasets can learn comprehensive features by utilizing the preliminary features from transfer learning. However, the performance is not guaranteed when taking these features to construct a new model, as the differences always exist between the source and target domains. In this paper, we propose to build an Evolution Programming-based framework to address various challenges. This framework automates both the feature learning and model building processes. It first identifies the most valuable features from pre-trained models and then constructs a suitable model to understand the characteristic features for different tasks. Each model differs in numerous ways. Overall, the experimental results effectively reach optimal solutions, demonstrating that a time-consuming task could also be conducted by an automated process that exceeds the human ability.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Back, T. (1996). Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms. Oxford: Oxford University Press. Back, T. (1996). Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms. Oxford: Oxford University Press.
Zurück zum Zitat Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(Feb), 281–305. Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(Feb), 281–305.
Zurück zum Zitat Chang, W.L. (2019). The impact of emotion: a blended model to estimate influence on social media. Information Systems Frontiers, 21(5), 1137–1151.CrossRef Chang, W.L. (2019). The impact of emotion: a blended model to estimate influence on social media. Information Systems Frontiers, 21(5), 1137–1151.CrossRef
Zurück zum Zitat Chen, C., Zhu, Q., Lin, L., & Shyu, M.L. (2013). Web media semantic concept retrieval via tag removal and model fusion. ACM Transactions on Intelligent Systems and Technology, 4(4), 61. Chen, C., Zhu, Q., Lin, L., & Shyu, M.L. (2013). Web media semantic concept retrieval via tag removal and model fusion. ACM Transactions on Intelligent Systems and Technology, 4(4), 61.
Zurück zum Zitat Chen, S.C., & Kashyap, R.L. (2001). A spatio-temporal semantic model for multimedia database systems and multimedia information systems. IEEE Transactions on Knowledge and Data Engineering, 13(4), 607–622.CrossRef Chen, S.C., & Kashyap, R.L. (2001). A spatio-temporal semantic model for multimedia database systems and multimedia information systems. IEEE Transactions on Knowledge and Data Engineering, 13(4), 607–622.CrossRef
Zurück zum Zitat Chen, S.C., Rubin, S.H., Shyu, M.L., & Zhang, C. (2006). A dynamic user concept pattern learning framework for content-based image retrieval. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 36(6), 772–783.CrossRef Chen, S.C., Rubin, S.H., Shyu, M.L., & Zhang, C. (2006). A dynamic user concept pattern learning framework for content-based image retrieval. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 36(6), 772–783.CrossRef
Zurück zum Zitat Chen, S.C., Shyu, M.L., & Kashyap, R. (1998). Augmented transition network as a semantic model for video data. International Journal of Networking and Information Systems, 3(3), 9–25. Chen, S.C., Shyu, M.L., & Kashyap, R. (1998). Augmented transition network as a semantic model for video data. International Journal of Networking and Information Systems, 3(3), 9–25.
Zurück zum Zitat Chen, S.C., Shyu, M.L., & Zhang, C. (2005). Innovative shot boundary detection for video indexing. In Video data management and information retrieval. IGI Global (pp. 217–236). Chen, S.C., Shyu, M.L., & Zhang, C. (2005). Innovative shot boundary detection for video indexing. In Video data management and information retrieval. IGI Global (pp. 217–236).
Zurück zum Zitat Chen, S.C., Shyu, M.L., Zhang, C., & Kashyap, R.L. (2001). Identifying overlapped objects for video indexing and modeling in multimedia database systems. International Journal on Artificial Intelligence Tools, 10(04), 715–734.CrossRef Chen, S.C., Shyu, M.L., Zhang, C., & Kashyap, R.L. (2001). Identifying overlapped objects for video indexing and modeling in multimedia database systems. International Journal on Artificial Intelligence Tools, 10(04), 715–734.CrossRef
Zurück zum Zitat He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In The IEEE conference on computer vision and pattern recognition (pp. 770–778). He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In The IEEE conference on computer vision and pattern recognition (pp. 770–778).
Zurück zum Zitat Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). Mobilenets: efficient convolutional neural networks for mobile vision applications. CoRR arXiv:1704.04861. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). Mobilenets: efficient convolutional neural networks for mobile vision applications. CoRR arXiv:1704.​04861.
Zurück zum Zitat Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K.Q. (2017). Densely connected convolutional networks. In The IEEE conference on computer vision and pattern recognition (pp. 4700–4708). Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K.Q. (2017). Densely connected convolutional networks. In The IEEE conference on computer vision and pattern recognition (pp. 4700–4708).
Zurück zum Zitat Lameski, P., Zdravevski, E., Mingov, R., & Kulakov, A. (2015). SVM parameter tuning with grid search and its impact on reduction of model over-fitting. In Rough sets, fuzzy sets, data mining, and granular computing (pp. 464–474): Springer. Lameski, P., Zdravevski, E., Mingov, R., & Kulakov, A. (2015). SVM parameter tuning with grid search and its impact on reduction of model over-fitting. In Rough sets, fuzzy sets, data mining, and granular computing (pp. 464–474): Springer.
Zurück zum Zitat Li, H., He, F., Liang, Y., & Quan, Q. (2019). A dividing-based many-objective evolutionary algorithm for large-scale feature selection. Soft Computing, 1–20. Li, H., He, F., Liang, Y., & Quan, Q. (2019). A dividing-based many-objective evolutionary algorithm for large-scale feature selection. Soft Computing, 1–20.
Zurück zum Zitat Li, X., Chen, S.C., Shyu, M.L., & Furht, B. (2002). Image retrieval by color, texture, and spatial information. In The 8th international conference on distributed multimedia systems (pp. 152–159). Li, X., Chen, S.C., Shyu, M.L., & Furht, B. (2002). Image retrieval by color, texture, and spatial information. In The 8th international conference on distributed multimedia systems (pp. 152–159).
Zurück zum Zitat Lin, L., & Shyu, M.L. (2010). Weighted association rule mining for video semantic detection. International Journal of Multimedia Data Engineering and Management (IJMDEM), 1(1), 37–54.CrossRef Lin, L., & Shyu, M.L. (2010). Weighted association rule mining for video semantic detection. International Journal of Multimedia Data Engineering and Management (IJMDEM), 1(1), 37–54.CrossRef
Zurück zum Zitat Mania, H., Guy, A., & Recht, B. (2018). Simple random search provides a competitive approach to reinforcement learning. CoRR arXiv:1803.07055. Mania, H., Guy, A., & Recht, B. (2018). Simple random search provides a competitive approach to reinforcement learning. CoRR arXiv:1803.​07055.
Zurück zum Zitat Mitchell, T.M. (1997). Machine learning. Mitchell, T.M. (1997). Machine learning.
Zurück zum Zitat Molchanov, P., Tyree, S., Karras, T., Aila, T., & Kautz, J. (2016). Pruning convolutional neural networks for resource efficient transfer learning. CoRR arXiv:1611.06440. Molchanov, P., Tyree, S., Karras, T., Aila, T., & Kautz, J. (2016). Pruning convolutional neural networks for resource efficient transfer learning. CoRR arXiv:1611.​06440.
Zurück zum Zitat Mukherjee, S. (2020). Emerging frontiers in smart environment and healthcare–A vision. Information Systems Frontiers, 22(1), 23–27.CrossRef Mukherjee, S. (2020). Emerging frontiers in smart environment and healthcare–A vision. Information Systems Frontiers, 22(1), 23–27.CrossRef
Zurück zum Zitat Pan, S.J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.CrossRef Pan, S.J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.CrossRef
Zurück zum Zitat Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Zurück zum Zitat Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M.P., Shyu, M.L., Chen, S.C., & Iyengar, S. (2018). A survey on deep learning: algorithms, techniques, and applications. ACM Computing Surveys, 51(5), 92. Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M.P., Shyu, M.L., Chen, S.C., & Iyengar, S. (2018). A survey on deep learning: algorithms, techniques, and applications. ACM Computing Surveys, 51(5), 92.
Zurück zum Zitat Pouyanfar, S., Tao, Y., Tian, H., Chen, S.C., & Shyu, M.L. (2019). Multimodal deep learning based on multiple correspondence analysis for disaster management. World Wide Web, 22(5), 1893–1911.CrossRef Pouyanfar, S., Tao, Y., Tian, H., Chen, S.C., & Shyu, M.L. (2019). Multimodal deep learning based on multiple correspondence analysis for disaster management. World Wide Web, 22(5), 1893–1911.CrossRef
Zurück zum Zitat Pouyanfar, S., Yang, Y., Chen, S.C., Shyu, M.L., & Iyengar, S. (2018). Multimedia big data analytics: a survey. ACM Computing Surveys, 51(1), 10.CrossRef Pouyanfar, S., Yang, Y., Chen, S.C., Shyu, M.L., & Iyengar, S. (2018). Multimedia big data analytics: a survey. ACM Computing Surveys, 51(1), 10.CrossRef
Zurück zum Zitat Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R.M. (2016). Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5), 1285–1298.CrossRef Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R.M. (2016). Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5), 1285–1298.CrossRef
Zurück zum Zitat Snoek, J., Larochelle, H., & Adams, R.P. (2012). Practical Bayesian optimization of machine learning algorithms. In Advances in neural information processing systems (pp. 2951–2959). Snoek, J., Larochelle, H., & Adams, R.P. (2012). Practical Bayesian optimization of machine learning algorithms. In Advances in neural information processing systems (pp. 2951–2959).
Zurück zum Zitat Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In The IEEE conference on computer vision and pattern recognition (pp. 2818–2826). Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In The IEEE conference on computer vision and pattern recognition (pp. 2818–2826).
Zurück zum Zitat Tian, H., & Chen, S.C. (2017). Mca-nn: multiple correspondence analysis based neural network for disaster information detection. In IEEE third international conference on multimedia big data (bigMM) (pp. 268–275): IEEE. Tian, H., & Chen, S.C. (2017). Mca-nn: multiple correspondence analysis based neural network for disaster information detection. In IEEE third international conference on multimedia big data (bigMM) (pp. 268–275): IEEE.
Zurück zum Zitat Tian, H., Chen, S.C., & Shyu, M.L. (2019). Genetic algorithm based deep learning model selection for visual data classification. In The 20th international conference on information reuse and integration for data science (pp. 127–134): IEEE. Tian, H., Chen, S.C., & Shyu, M.L. (2019). Genetic algorithm based deep learning model selection for visual data classification. In The 20th international conference on information reuse and integration for data science (pp. 127–134): IEEE.
Zurück zum Zitat Tian, H., Chen, S.C., Shyu, M.L., & Rubin, S. (2019). Automated neural network construction with similarity sensitive evolutionary algorithms. In The 20th international conference on information reuse and integration for data science (pp. 283–290): IEEE. Tian, H., Chen, S.C., Shyu, M.L., & Rubin, S. (2019). Automated neural network construction with similarity sensitive evolutionary algorithms. In The 20th international conference on information reuse and integration for data science (pp. 283–290): IEEE.
Zurück zum Zitat Tian, H., Pouyanfar, S., Chen, J., Chen, S.C., & Iyengar, S.S. (2018). Automatic convolutional neural network selection for image classification using genetic algorithms. In The IEEE international conference on information reuse and integration (pp. 444–451): IEEE. Tian, H., Pouyanfar, S., Chen, J., Chen, S.C., & Iyengar, S.S. (2018). Automatic convolutional neural network selection for image classification using genetic algorithms. In The IEEE international conference on information reuse and integration (pp. 444–451): IEEE.
Zurück zum Zitat Tian, H., Tao, Y., Pouyanfar, S., Chen, S.C., & Shyu, M.L. (2019). Multimodal deep representation learning for video classification. World Wide Web, 22(3), 1325–1341.CrossRef Tian, H., Tao, Y., Pouyanfar, S., Chen, S.C., & Shyu, M.L. (2019). Multimodal deep representation learning for video classification. World Wide Web, 22(3), 1325–1341.CrossRef
Zurück zum Zitat Tian, H., Zheng, H.C., & Chen, S.C. (2018). Sequential deep learning for disaster-related video classification. In IEEE conference on multimedia information processing and retrieval (MIPR) (pp. 106–111): IEEE. Tian, H., Zheng, H.C., & Chen, S.C. (2018). Sequential deep learning for disaster-related video classification. In IEEE conference on multimedia information processing and retrieval (MIPR) (pp. 106–111): IEEE.
Zurück zum Zitat Wang, D., & Zheng, T.F. (2015). Transfer learning for speech and language processing. In The asia-pacific signal and information processing association annual summit and conference (pp. 1225–1237): IEEE. Wang, D., & Zheng, T.F. (2015). Transfer learning for speech and language processing. In The asia-pacific signal and information processing association annual summit and conference (pp. 1225–1237): IEEE.
Zurück zum Zitat Winston, P.H. (1992). Artificial intelligence. Winston, P.H. (1992). Artificial intelligence.
Zurück zum Zitat Yang, J., & Honavar, V. (1998). Feature subset selection using a genetic algorithm. In Feature extraction, construction and selection (pp. 117–136): Springer. Yang, J., & Honavar, V. (1998). Feature subset selection using a genetic algorithm. In Feature extraction, construction and selection (pp. 117–136): Springer.
Zurück zum Zitat Yao, X., Liu, Y., & Lin, G. (1999). Evolutionary programming made faster. IEEE Transactions on Evolutionary Computation, 3(2), 82–102.CrossRef Yao, X., Liu, Y., & Lin, G. (1999). Evolutionary programming made faster. IEEE Transactions on Evolutionary Computation, 3(2), 82–102.CrossRef
Zurück zum Zitat Zhu, Q., Lin, L., Shyu, M.L., & Chen, S.C. (2011). Effective supervised discretization for classification based on correlation maximization. In IEEE international conference on information reuse & integration (pp. 390–395): IEEE. Zhu, Q., Lin, L., Shyu, M.L., & Chen, S.C. (2011). Effective supervised discretization for classification based on correlation maximization. In IEEE international conference on information reuse & integration (pp. 390–395): IEEE.
Zurück zum Zitat Zhu, W., Cui, P., Wang, Z., & Hua, G. (2015). Multimedia big data computing. IEEE Multimedia, 22(3), 96–c3.CrossRef Zhu, W., Cui, P., Wang, Z., & Hua, G. (2015). Multimedia big data computing. IEEE Multimedia, 22(3), 96–c3.CrossRef
Metadaten
Titel
Evolutionary Programming Based Deep Learning Feature Selection and Network Construction for Visual Data Classification
Publikationsdatum
25.06.2020
Erschienen in
Information Systems Frontiers / Ausgabe 5/2020
Print ISSN: 1387-3326
Elektronische ISSN: 1572-9419
DOI
https://doi.org/10.1007/s10796-020-10023-6

Weitere Artikel der Ausgabe 5/2020

Information Systems Frontiers 5/2020 Zur Ausgabe

Premium Partner