Abstract
Machine learning and knowledge acquisition from experts have distinct capabilities that appear to complement one another. We report a study that demonstrates the integration of these approaches can both improve the accuracy of the developed knowledge base and reduce development time. In addition, we found that users expected the expert systems created through the integrated approach to have higher accuracy than those created without machine learning and rated the integrated approach less difficult to use. They also provided favorable evaluations of both the specific integrated software, a system called The Knowledge Factory, and of the general value of machine learning for knowledge acquisition.
Article PDF
Similar content being viewed by others
References
Attar Software (1989). Structured decision tasks methodology for developing and integrating knowledge base systems. Attar Software, Leigh, Lancashire.
Boose, J. H. (1986). ETS: A system for the transfer of human expertise. In J. S. Kowalik (Ed.), Knowledge based problem solving. Prentice-Hall: New York.
Buntine, W., & Stirling, D. (1991). Interactive induction. In J. E. Hayes, D. Michie, & Ė. Tyugu (Eds.), Machine Intelligence 12. Clarendon Press: Oxford.
Compton, P., Edwards, G., Srinivasan, A., Malor, R., Preston, P., Kang, B., & Lazarus, L. (1992). Ripple down rules: Turning knowledge acquisition into knowledge maintenance. Artificial Intelligence in Medicine, 4: 47–59.
Davis, R., & Lenat, D. B. (1982). Knowledge-based systems in artificial intelligence. McGraw-Hill: New York.
De Raedt, L. (1992). Interactive theory revision. Academic Press: London.
Gams, M., Drobnič, M., & Karba, N. (1996). Average-case improvements when integrating ML and KA. Applied Intelligence, 6: 87–99.
Kodratoff, Y., & Vrain, C. (1993). Acquiring first order knowledge about air traffic control. Knowledge Acquisition, 5: 1–36.
Langley, P. (1996). Elements of Machine Learning. Morgan Kaufmann: San Francisco.
Le Grand, A., & Sallantin, J. (1994). A framework to improve knowledge acquisition based on machine learning. Proceedings of the Eleventh European Conference on Artificial Intelligence (pp. 493–497). John Wiley: London.
Merz, C. J., & Murphy, P. M. (1997). UCI repository of machine learning databases. [Machine-readable data repository]. University of California, Department of Information and Computer Science, Irvine.
Michalski, R. S. (1983). A theory and methodology of inductive learning. In R. S. Michalski, J. G. Carbonell, & T. M. Mitchell (Eds.), Machine learning: An artificial intelligence approach. Springer-Verlag: Berlin.
Morik, K. (1987). Aquiring domain models. International Journal of Man-Machine Studies, 26: 93–104.
Morik, K., Wrobel, S., Kietz, J.-U., & Emde, W. (1993). Knowledge acquisition and machine learning: Theory, methods, and applications. Academic Press: London.
Nedellec, C., & Causse, K. (1992). Knowledge refinement using knowledge acquisition and machine learning methods. Proceedings of the 1992 European Knowledge Acquisition Workshop (pp. 171–190). Springer-Verlag: Berlin.
Nedellec, C., Correia, J., Ferreira, J. L., & Costa, E. (1994). Machine learning goes to the bank. Applied Artificial Intelligence, 8: 593–615.
O'Neil, J. L., & Pearson, R. A. (1987). A development environment for inductive learning systems. Proceedings of the 1987 Australian Joint Artificial Intelligence Conference (pp. 673–680). Sydney.
Quinlan, J. R. (1993). C4.5: Programs for machine learning. Morgan Kaufmann: San Mateo, CA.
Schmalhofer, F., & Tschaitschian, B. (1995). Cooperative knowledge evolution for complex domains. In G. Tecuci & Y. Kodratoff (Eds.), Machine learning and knowledge acquisition: Integrated approaches. Academic Press, London.
Smith, R. G., Winston, H. A., Mitchell, T. M., & Buchanan, B. G. (1985). Representation and use of explicit justifications for knowledge base refinement. Proceedings of the Ninth International Joint Conference on Artificial Intelligence (pp. 673–680). Morgan Kaufmann, San Mateo, Ca.
Sommer, E., Morik, K., André´e, J.-M., & Uszynski, M. (1994). What online machine learning can do for knowledge acquisition—A case study. Knowledge Acquisition, 6: 435–460.
Tecuci, G., & Kodratoff, Y. (1990). Apprenticship learning in imperfect domain theories. In Y. Kodratoff & R. Michalski (Eds.), Machine learning: An artificial intelligence approach. Morgan Kaufmann: San Mateo, CA.
Tecuci, G. (1995). Building knowledge bases through multistrategy learning and knowledge acquisition. In G. Tecuci & Y. Kodratoff (Eds.), Machine learning and knowledge acquisition: Integrated approaches. Academic Press: London.
Webb, G. I. (1992). Man-machine collaboration for knowledge acquisition. Proceedings of the Fifth Australian Joint Conference on Artificial Intelligence (pp. 329–334). World Scientific: Singapore.
Webb, G. I. (1993). DLGref2: Techniques for inductive knowledge refinement. Proceedings of the IJCAIWorkshop on Machine Learning and Knowledge Acquisition: Common Issues, Contrasting Methods, and Integrated Approaches (pp. 236–252). Chambery, France.
Webb, G. I. (1996). Integrating machine learning with knowledge acquisition through direct interaction with domain experts. Knowledge-Based Systems, 9: 253–266.
Webb, G. I., & Agar, J. W. M. (1992). Inducing diagnostic rules for glomerular disease with the DLG machine learning algorithm. Artificial Intelligence in Medicine, 4: 3–14.
Webb, G. I., & Wells, J. (1995). Recent progress in machine-expert collaboration for knowledge acquisition. Proceedings of the Eighth Australian Joint Conference on Artificial Intelligence (pp. 291–298). World Scientific: Singapore.
Webb, G. I., & Wells, J. (1996). Experimental evaluation of integrating machine learning with knowledge acquisition through direct interaction with domain experts. Proceedings of the Pacific Knowledge Acquisition Workshop (pp. 170–189). Sydney, Australia.
Wilkins, D. C. (1988). Knowledge base refinement using apprenticeship learning techniques. Proceedings of the Seventh National Conference on Artificial Intelligence (pp. 646–651). Morgan Kaufmann: San Mateo, CA.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Webb, G.I., Wells, J. & Zheng, Z. An Experimental Evaluation of Integrating Machine Learning with Knowledge Acquisition. Machine Learning 35, 5–23 (1999). https://doi.org/10.1023/A:1007504102006
Issue Date:
DOI: https://doi.org/10.1023/A:1007504102006