nach oben

Erschienen in:

2018 | OriginalPaper | Buchkapitel

2. Meta-heuristics, Machine Learning, and Deep Learning Methods

verfasst von : Hitoshi Iba

Erschienen in: Evolutionary Approach to Machine Learning and Deep Neural Networks

Verlag: Springer Singapore

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This chapter introduces several meta-heuristics and learning methods, which will be employed in later chapters. These methods will be employed to extend evolutionary computation frameworks in later chapters. Readers familiar with these methods may skip this chapter.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Introduction

Nächstes Kapitel Evolutionary Approach to Deep Learning

Given a set of vectors \(\mathbf {v}_1, \ldots , \mathbf {v}_n\), its Gram matrix G is an \(n\times n\) matrix, whose element is an inner product of two vectors, i.e., \(G_{ij} = \mathbf {v}_i \cdot \mathbf {v}_j\).

The point P is in the range which is outside a circle (the center is a mid-point of centers of circle \(O_{1}\) and circle \(O_{2}\), and the radius is \(\frac{\mid r_{1} - r_{2}\mid }{2}\)), and which is inside a circle (the center is the same, and the radius is\(\frac{(r_{1} +r_{2})}{2}\)).

A degree measure of the similarity between two documents A and B, i.e., \(\frac{\mid A \cap B\mid }{\mid A \cup B \mid }\).

The used parameters are as follows: 300 sample points, cluster centers \(=\) [[2, 2], [−2, −2], [2, −2]], and cluster std. \(=\) 0.5.

The first, second, and fifth of these layers are linked to max pooling.

The softmax activation function is defined as \(f_i(x_1,\ldots ,x_d) = \frac{\exp (x_i)}{\sum _j \exp (x_j)}\;(i=1,\ldots ,d)\), where d is the unit number of the layer.

Principal component analysis of the RGB values in the data set is performed and then random changes are made, centered on these axes, thereby preserving the essential features of the original natural images.

ILSVRC2012 dataset is a subset of the ImageNet database [40]. It contains 1,000 object categories. The training set, validation set, and testing set contain 1.3 M, 50 K, and 150 K images, respectively. The input images are of \(224\times 224\times 3\) pixels.

Aha, D.W., Kibler, D., Albert, M.: Instance-based learning algorithms. Mach. Learn. 6, 37–66 (1991)

Angeline, P.J.: Evolutionary optimization versus particle swarm optimization: philosophy and performance differences. In: Porto, V.W., Saravanan, N., Waagen, D., Eiben, A.E. (eds.) Evolutionary Programming VII, pp. 601–610. Springer, Berlin (1998)

Bäck, T., Fogel, D.B., Michalewicz, Z. (eds.): Evolutionary Computation 1: Basic Algorithms and Operators. Institute of Physics Publishing, Bristol (2000)

Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Berlin (2006)

Buchberger, B.: A criterion for detecting unnecessary reductions in the construction of Gröbner-bases. In: Proceedings of the International Symposium on Symbolic and Algebraic Computation (EUROSAM’79), pp. 3–21 (1979)CrossRef

Buchberger, B.: A note on the complexity of constructing Gröbner-bases. In: van Hulzen, J.A. (ed.) Computer Algebra, EUROCAL 1983. Lecture Notes in Computer Science, vol. 162. Springer, Berlin (1983)

Buchberger, B.: A critical pair completion algorithm for finitely generated ideals in rings. In: Borger, E., Hasenjaeger, G., Rodding, D. (eds.) Logic and Machines: Decision Problems and Complexity. Lecture Notes in Computer Science, vol. 171. Springer, Berlin (1984)CrossRef

Buchberger, B.: Gröbner bases: an algorithmic method in polynomial ideal theory. In: Bose, N.K. (ed.) Multidimensional Systems Theory. D. Reidel Publishing Company, Dordrecht (1985)CrossRef

Buchberger, B.: Applications of Gröbner bases in non-linear computational geometry. In: Jansen, R. (ed.) Trends in Computer Algebra. Lecture Notes in Computer Science, vol. 296. Springer, Berlin (1987)

10.

Buchberger, B., Winkler, F.: Gröbner Bases and Applications, vol. 251. Cambridge University Press, Cambridge (1998)

11.

Caviness, B.F., Johnson, J.R. (eds.): Quantifier Elimination and Cylindrical Algebraic Decomposition. Springer, Berlin (2013)

12.

Chakraborti, N., Misra, K., Bhatt, P., Barman, N., Prasad, R.: Tight-binding calculations of Si-H clusters using genetic algorithms and related techniques: studies using differential evolution. J. Phase Equilibria 22(5), 525–530 (2001)CrossRef

13.

Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)CrossRef

14.

Chou, S., Schelter, W.F.: Proving geometry theorems with rewrite rules. J. Autom. Reason. 2, 253–273 (1986)CrossRefMATH

15.

Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)MATH

16.

Dasarathy, B.: Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos (1991)

17.

Drucker, H.: Improving Regression Using Boosting Techniques. In: Proceedings of International Conference on Machine Learning (ICML97) (1997)

18.

Dueck, D., Frey, B.J.: Non-metric affinity propagation for unsupervised image categorization. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8. IEEE (2007)

19.

Eberhart, R.C., Shi, Y.: Comparison between genetic algorithms and particle swarm optimization. In: Proceedings of the Seventh Annual Conference on Evolutionary Programming, pp. 611–619 (1998)

20.

Fan, R.-E., Chang, L.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)MATH

21.

Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70, 41–54 (2006)CrossRef

22.

Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proceedings of International Conference on Machine Learning (ICML96) (1996)

23.

Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315, 972–976 (2007)MathSciNetCrossRefMATH

24.

Fukushima, K.: Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980)CrossRefMATH

25.

Gämperle, R., Müller, S.D., Koumoutsakos, P.: A parameter study for differential evolution. In: Proceedings of International Conference on Advances in Intelligent Systems, Fuzzy Systems, Evolutionary Computation, pp. 293–298 (2002)

26.

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

27.

Higashi, N., Iba, H.: Particle swarm optimization with Gaussian mutation. In: Proceedings of IEEE Swarm Intelligence Symposium (SIS03), pp. 72–79 (2003)

28.

Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)MathSciNetCrossRefMATH

29.

Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press (1975)

30.

Hu, R., Namee, B.M., Delany, S.J.: Off to a good start: using clustering to select the initial training set in active learning. In: Proceedings of the Florida Artificial Intelligence Research Society Conference (FLAIRS) (2010)

31.

Huang, C.-H., Wang, C.-J.: A GA-based feature selection and parameters optimization for support vector machines. Expert Syst. Appl. 31, 231–240 (2006)CrossRef

32.

Iba, H., Hirochika Inoue, H.: Reasoning of geometric concepts based on algebraic constraint-directed method. In: Proceedings of the IJCAI, pp. 143–151 (1991)

33.

Iba, H., Noman, N.: New Frontiers in Evolutionary Algorithms: Theory and Applications. World Scientific Publishing Company, Singapore (2011)

34.

Jia, Y., Wang, J., Zhang, C., Hua, X.-S.: Finding image exemplars using fast sparse affinity propagation. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 639–642. ACM (2008)

35.

Kapur, D.: Geometric reasoning and artificial intelligence: introduction to the special volume. Artif. Intell. 37 (1988)CrossRef

36.

Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of IEEE the International Conference on Neural Networks, pp. 1942–1948 (1995)

37.

Kennedy, J., Eberhart, R.C.: Swarm Intelligence. Morgan Kaufmann Publishers, San Francisco (2001)

38.

Kennedy, J., Spears, W.M.: Matching algorithms to problems: an experimental test of the particle swarm and some genetic algorithms on the multimodal problem generator. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp. 78–83 (1998)

39.

Krink, T., Filipič, B., Fogel, G., Thomsen, R.: Noisy optimization problems – a particular challenge for differential evolution? In: Proceedings of Congress on Evolutionary Computation, pp. 332–339 (2004)

40.

Krizhevsky, A., Sutskerver, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25 (NIPS), pp. 1097–1105 (2012)

41.

Larsen, A.B.L., S\(\phi \)nderby, S.K., Winther, O.: Autoencoding beyond pixels using a learned similarity metric (2015). arXiv:1512.09300

42.

Laubenbacher, R., Stigler, B.: A computational algebra approach to the reverse engineering of gene regulatory networks. J. Theor. Biol. 229, 523–537 (2004)MathSciNetCrossRef

43.

Le, Q., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G., Dean, J., Ng, A.: Building high-level features using large scale unsupervised learning. In: Proceedings of the 29th International Conference on Machine Learning (2012)

44.

LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRef

45.

Limbeck, J.: Computation of approximate border bases and applications, Ph.D. thesis, Passau, Universität Passau, Dissertation (2014)

46.

Liu, Z., Li, P., Zheng, Y., Sun, M.: Clustering to find exemplar terms for key phrase extraction. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, pp. 257–266. Association for Computational Linguistics (2009)

47.

Loos, R.: Introduction. In: Buchberger, B., et al. (eds.) Computer Algebra Symbolic and Algebraic Computation. Springer, Berlin (1982)

48.

Maclin, R., Opitz, D.: An empirical evaluation of bagging and boosting. In: Proceedings of National Conference on Artificial Intelligence (AAAI97) (1997)

49.

Möller, H.M., Buchberger, B.: The construction of multivariate polynomials with preassigned zeros. In: Proceedings of the European Computer Algebra Conference on Computer Algebra, pp. 24–31 (1982)

50.

Mourrain, B.: A new criterion for normal form algorithms. Applied Algebra, Algebraic Algorithms and Error-correcting Codes, pp. 430–442. Springer, Berlin (1999)CrossRefMATH

51.

Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 427–436 (2015)

52.

Noman, N., Iba, H.: Enhancing differential evolution performance with local search for high dimensional function optimization. In: Proceedings of Genetic and Evolutionary Computation Conference (GECCO2005), pp. 967–974 (2005)

53.

Noman, N., Iba, H.: A new generation alternation model for differential evolution. In: Proceedings of Genetic and Evolutionary Computation Conference (GECCO 2006), pp. 1265–1272 (2006)

54.

Noman, N., Iba, H.: Differential evolution for economic load dispatch problems. Elsevier Electric Power Syst. Res. 78(8), 1322–1331 (2008)CrossRef

55.

Noman, N., Iba, H.: Accelerating differential evolution using an adaptive local search. IEEE Trans. Evol. Comput. 12(1), 107–125 (2008)CrossRef

56.

Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)CrossRef

57.

Paul, T.K., Ueno, K., Iwata, K., Hayashi, T., Honda, N.: Genetic algorithm based methods for identification of health risk factors aimed at preventing metabolic syndrome. In: Proceedings of the 7th International Conference on Simulated Evolution And Learning (SEAL’08). LNCS, vol. 5361, pp. 210–219. Springer, Berlin (2008)

58.

Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Mateo (1988)

59.

Price, K.V., Storn, R.M., Lampinen, J.A.: Differential Evolution: A Practical Approach to Global Optimization. Springer, Berlin (2005)MATH

60.

Quinlan, J.R.: Bagging, Boosting, and C4.5. In: Proceedings of National Conference on Artificial Intelligence (AAAI96) (1996)

61.

Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks (2015). arXiv:1511.06434

62.

Sato, H., Hasegawa, Y., Bollegala, D., Iba, H.: Improved sampling using loopy belief propagation for probabilistic model building genetic programming. Swarm Evol. Comput. 23, 1–10 (2015)CrossRef

63.

Stifter, S.: Algebraic methods for computing inverse kinematics. J. Intell. Robot. Syst. 11(1–2), 79–89 (1994)CrossRefMATH

64.

Storn, R.: System design by constraint adaptation and differential evolution. IEEE Trans. Evol. Comput. 3(1), 22–34 (1999)CrossRef

65.

Storn, R., Price, K.V.: Differential evolution – a simple and efficient adaptive scheme for global optimization over continuous spaces. Technical report TR-95-012, ICSI (1995)

66.

Storn, R., Price, K.V.: Differential evolution -a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)MathSciNetCrossRefMATH

67.

Swan, J., Neumann, G.K., Krawiec, K.: Analysis of semantic building blocks via Grobner bases. In: Johnson, C., Krawiec, K., O’Neill, M., Moraglio, A. (eds.) Semantic Methods in Genetic Programming (SMGP) at Parallel Problem Solving from Nature (PPSN XIV), Ljubljana, Slovenia (2014)

68.

Tipping, M.E.: The relevance vector machine. Advances in Neural Information Processing Systems, pp. 652–658. MIT Press, Cambridge (2000)

69.

Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

70.

Yang, Y., Iba, H.: Fooling voice based on evolutionary computation. In: Proceedings of Evolutionary Computation Symposium, Dec. 9–10, Hokkaido, Japan (2017)

71.

Zaharie, D.: Critical values for the control parameters of differential evolution algorithms. In: Proceedings of MENDEL 2002, 8th International Conference on Soft Computing, pp. 62–67 (2002)

72.

Zha, Z.-J., Yang, L., Mei, T., Wang, M., Wang, Z.: Visual query suggestion. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 15–24. ACM (2009)

Titel: Meta-heuristics, Machine Learning, and Deep Learning Methods
verfasst von: Hitoshi Iba
Verlag: Springer Singapore
Buch: Evolutionary Approach to Machine Learning and Deep Neural Networks
Print ISBN: 978-981-13-0199-5

Electronic ISBN: 978-981-13-0200-8

Copyright-Jahr: 2018
DOI: https://doi.org/10.1007/978-981-13-0200-8_2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner