Abstract
Supervised learning in attribute-based spaces is one of the most popular machine learning problems studied and, consequently, has attracted considerable attention of the genetic algorithm community. The fullmemory approach developed here uses the same high-level descriptive language that is used in rule-based systems. This allows for an easy utilization of inference rules of the well-known inductive learning methodology, which replace the traditional domain-independent operators and make the search task-specific. Moreover, a closer relationship between the underlying task and the processing mechanisms provides a setting for an application of more powerful task-specific heuristics. Initial results obtained with a prototype implementation for the simplest case of single concepts indicate that genetic algorithms can be effectively used to process high-level concepts and incorporate task-specific knowledge. The method of abstracting the genetic algorithm to the problem level, described here for the supervised inductive learning, can be also extended to other domains and tasks, since it provides a framework for combining recently popular genetic algorithm methods with traditional problem-solving methodologies. Moreover, in this particular case, it provides a very powerful tool enabling study of the widely accepted but not so well understood inductive learning methodology.
Article PDF
Similar content being viewed by others
References
Antonisse, H.J., & Keller, K.S. (1987). Genetic operators for high level knowledge representation.Proceedings of the Second International Conference on Genetic Algorithms (pp. 69–76). Hillsdale, NJ: Lawrence Erlbaum Associates.
Baker, J.E. (1987). Reducing bias and inefficiency in the selection algorithm.Proceedings of the Second International Conference on Genetic Algorithms (pp. 14–21). Hillsdale, NJ: Lawrence Erlbaum Associates.
Booker, L.B. (1989). Triggered rule discovery in classifier system.Proceedings of the Third International Conference on Genetic Algorithms (pp. 265–174). San Mateo, CA: Morgan Kaufmann.
Clark, P., & Niblett, T. (1987). Induction in noisy domains. In I. Bratko & N. Lavrac (Eds.),Progress in machine learning. Sigma Press.
Davis, L. (Ed). (1991).Handbook of genetic algorithms. New York: Van Nostrand Reinhold.
DeJong, K.A. (1985). Genetic algorithms: A 10 year perspective.Proceedings of the First International Conference on Genetic Algorithms (pp. 169–177). Hillsdale, NJ: Lawrence Erlbaum Associates.
DeJong, K.A. (1988). Learning with genetic algorithm: An overview.Machine Learning, 3 (2/3), 121–138.
DeJong, K.A. (1990). Genetic-algorithm-based learning. In Y. Kondratoff & R.S. Michalski (Eds.),Machine learning: An artificial intelligence approach. San Mateo, CA: Morgan Kaufmann.
DeJong, K.A., & Spears, W.M. (1991). Learning concept classification rules using genetic algorithms.Proceedings of the International Joint Conference on Artificial Intelligence (pp. 651–656). San Mateo, CA: Morgan Kaufmann.
Forrest, S. (1985). Implementing semantic networks structures using the classifier system.Proceedings of the First International Conference on Genetic Algorithms (pp. 24–44). Hillsdale, NJ: Lawrence Erlbaum Associates.
Goldberg, D.E. (1985). Genetic algorithm and rule learning in dynamic system control.Proceedings of the First International Conference on Genetic Algorithms (pp. 8–15). Hillsdale, NJ: Lawrence Erlbaum Associates.
Goldberg, D.E. (1989).Genetic algorithms in search, optimization & machine learning. Reading, MA: Addison-Wesley.
Goldberg, D.E., & Holland, J.H. (1988). Genetic algorithms and machine learning.Machine Learning 3 (2/3), 95–99.
Grefenstette, J. (1987). Incorporating problem specific knowledge into genetic algorithms. In L. Davis (Ed.),Genetic algorithms and simulated annealing. London: Pitman.
Grefenstette, J. (1991). Lamarckian learning in multi-agent environments.Proceedings of the Fourth International Conference on Genetic Algorithms (pp. 303–310). San Mateo, CA: Morgan Kaufmann.
Green, D.P., & Smith, S.F. (1985). A genetic system for learning models of consumer choice.Proceedings of the Second International Conference on Genetic Algorithms (pp. 217–223). Hillsdale, NJ: Lawrence Erlbaum Associates.
Holland, J.H. (1975).Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.
Holland, J.H. (1986). Escaping brittleness. In R.S. Michalski, J. Carbonell & T. Mitchel (Eds.),Machine learning: An artificial intelligence approach (Vol. 2). San Mateo, CA: Morgan Kaufmann.
Holland, J.H., Holyoak, K.J., Nisbett, R.E., & Thagard, P.R. (1986).Induction. Cambridge, MA: MIT Press.
Janikow, C.Z. (1991).Inductive learning from attribute-based examples: A knowledge-intensive genetic algorithm approach. Doctoral dissertation, University of North Carolina at Chapel Hill, Department of Computer Science.
Janikow, C.Z. (in press). Some experiments with a stochastic production system for supervised inductive learning. InArtificial intelligence: Methodology, systems, applications (Vol. 5). Amsterdam: North-Holland.
Kaufman, K.A., Michalski, R.S., & Schultz, A.C. (1989).EMERALD 1: An integrated system of machine learning and discovery programs for education and research. (Technical Report MLI-89-12). Fairfax, VA: Center for AI, George Mason University.
Koza, J.R. (1989). Hierarchical genetic algorithms operating on populations of computer programs.Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (pp. 768–774). San Mateo, CA: Morgan Kaufmann.
Liepins, G.E., & Wang, L.A. (1991). Classifier system learning of Boolean concepts.Proceedings of the Fourth International Conference on Genetic Algorithms (pp. 318–323). San Mateo, CA: Morgan Kaufmann.
Michalewicz, Z., & Janikow, C.Z. (1991). Genetic algorithms for numerical optimization.Statistics and Computing, 1 (1), 75–89.
Michalski, R.S. (1983). Theory and methodology of inductive learning. In R.S. Michalski, J. Carbonell, & T. Mitchel (Eds.),Machine learning: An artificial intelligence approach (Vol. 1). San Mateo, CA: Morgan Kaufmann.
Michalski, R.S. (1986). Understanding the nature of learning. In R.S. Michalski, J. Carbonell, & T. Mitchell (Eds.),Machine learning: An artificial intelligence approach (Vol. 2). San Mateo, CA: Morgan Kaufmann.
Michalski, R.S., & Chilausky, R.L. (1980). Learning by being told and learning from examples: An experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis.International Journal of Policy Analysis and Information Systems, 4 (2), 125–161.
Michalski, R.S., Mozetic, I., Hong, J., & Lavrac, N. (1986).The AQ15 inductive learning system: An overview and experiments (Technical Report UIUCDCS-R-86-1260). Urbana, IL: Department of Computer Science, University of Illinois at Urbana-Champaign.
Quinlan, J.R. (1986). Induction of decision trees.Machine Learning, 1 (1), 81–106.
Quinlan, J.R. (1988). An empiricial comparison of genetic and decision-tree classifiers.Proceedings of the Fifth International Conference on Machine Learning (pp. 135–141). San Mateo, CA: Morgan Kaufmann.
Quinlan, J.R. (1990). Probabilistic decision trees. In Y. Kondratoff & R.S. Michalski (Eds.),Machine learning: An artificial intelligence approach. San Mateo, CA: Morgan Kaufmann.
Rendell, L.A. (1985). Genetic plans and the probabilistic learning system: Synthesis and results.Proceedings of the First International Conference on Genetic Algorithms (pp. 60–73). Hillsdale, NJ: Lawrence Erlbaum Associates.
Rendell, L., Cho, H., & Seshu, R. (1989). Improving the design of similarity-based rule-learning systems.International Journal of Expert Systems, 2 (1), 97–133.
Schaffer, J.D. (1985). Learning multiclass pattern discrimination.Proceedings of the First International Conference of Genetic Algorithms (pp. 74–79). Hillsdale, NJ: Lawrence Erlbaum Associates.
Sedbrook, T.A., Wright, H., & Wright, R. (1991). Application of genetic classifier for patient triage.Proceedings of the Fourth International Conference on Genetic Algorithms (pp. 334–338). San Mateo, CA: Morgan Kaufmann.
Smith, S.F. (1980).A learning system based on genetic algorithms. Doctoral dissertation, University of Pittsburgh, Department of Computer Science.
Spears, W.M., & DeJong, K.A. (1990). Using genetic algorithms for supervised concept learning.Proceedings of the Second International Conference on Tools for AI (pp. 335–341). Washington, DC: IEEE Computer Society Press.
Weiss, S.M., & Kapouleas, I. (1989). An empirical comparison of pattern recognition, neural nets, and machine learning classification methods.Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (pp. 781–787). San Mateo, CA: Morgan Kaufmann.
Weiss, S.M., & Kulikowski, C.A. (1990).Computer systems that learn. San Mateo, CA: Morgan Kaufmann.
Wilson, S. (1987). Classifier systems and the animat problem.Machine Learning, 2 (3), 199–228.
Wnek, J., & Michalski, R. (1991).Hypothesis-driven constructive induction in AQ17: A method and experiments (Technical Report MLI-91-4). Fairfax, VA: Center for AI, George Mason University.
Wnek, J., Sarma, J., Wahab, A., & Michalski, R.S. (1990). Comparing learning paradigms via diagramatic visualization. In M. Emrich, Z. Ras, & M. Zemankowa (Eds.),Methodologies for intelligent systems 5. Amsterdam: North-Holland.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Janikow, C.Z. A knowledge-intensive genetic algorithm for supervised learning. Mach Learn 13, 189–228 (1993). https://doi.org/10.1007/BF00993043
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00993043