Abstract
A large number of algorithms have been developed for classification and recognition. These algorithms can be divided into three major paradigms: statistical pattern recognition, neural networks, and model-based vision. Neural networks embody an especially rich field of approaches based on a variety of architectures, learning mechanisms, biological and algorithmic motivations, and application areas. Mathematical analysis of these approaches and paradigms reveals that there are only a few computational concepts permeating all the diverse approaches and serving as a basis for all paradigms and algorithms for classification and recognition.
These basic computational concepts are reviewed in this paper with the purposes of (i) providing a mathematical continuity to seemingly disparate techniques, (ii) establishing basic mathematical limitations on applicability of existing techniques, (iii) discerning fundamental questions facing the classification field, and (iv) searching for directions in which answers to these questions may be found.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M.L. Minsky and S.A. Papert,Perceptrons, MIT Press: Cambridge, MA, 1988.
R.O. Duda and H. Fossum, “Pattern classification by iteratively determined linear and piecewise linear discriminant functions,”IEEE Trans. Electron. Comput. vol. EC-15, pp. 220–232, 1966.
Y.C. Ho and A.K. Agrawala, “On pattern classification algorithms: Introduction and survey,”Proc. IEEE, vol. 56, pp. 2101–2114, 1968.
D.F. Specht, “Generation of polynomial discriminant functions for pattern recognition,”IEEE Trans. Electron. Comput., vol. EC-16, pp. 308–319, 1967.
N.J. Nilsson,Learning Machines, McGraw-Hill: New York, 1965.
B. Moore and T. Poggio, “Representation properties of multi-layer feedforward networks,” presented at 1st Annual International Neural Network Society Meeting, Boston, MA, 1988.
G.A. Carpenter, “Neural network models for pattern recognition and associative memory,”Neural Networks, vol. 2, pp. 243–257, 1989.
I.D. Longstaff and J.F. Cross, “A pattern recognition approach to understanding the multi-layer perception,”Pattern Recognition Letters, vol. 5, pp. 315–319, 1987.
K.J. Lang, A.H. Waibel, and G.E. Hinton, “A time-delay neural network architecture for isolated word recognition,”Neural Networks, vol. 3, pp. 23–44, 1990.
G.A. Carpenter and S. Grossberg, “A massively parallel architecture for a self-organizing neural pattern recognition machine,”Comput. Vis. Graph. Image Process., vol. 37, pp. 54–115, 1987.
A.D. Gordon,Classification, Chapman and Hall: London, 1981.
J.J. Hopfield, “Neural networks and physical systems with emergent collective computation abilities,”Proc. Nat. Acad. Sci. U.S.A., vol. 79, pp. 2554–2558, 1982.
D.L. Reilly, C. Scofield, C. Elbaum, and L.N. Cooper, “Learning systems architectures composed of multiple learning modules,” inProc. 1st International IEEE Conference on Neural Networks, 1987.
D.F. Specht, Probabilistic neural networks,”Neural Networks, vol. 3, pp. 109–118, 1990.
T. Kohonen, “Automatic formation of topological maps of patterns in a self organizing system,”Proc. 2nd Scandinavian Conference on Image Analysis, Espoo, Finland, 1981, pp. 214–220.
K. Fukushima, “Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position.”Biol. Cybernet., vol. 36, pp. 193–202, 1980.
T. Poggio, “Learning, regularization and splines,” presented at 1st Annual International Neural Network Society Meeting, Boston, MA, 1988.
P.K. Simpson,Artificial Neural Systems, Pergamon Press: New York, 1990.
K. Fukunaga,Introduction to Pattern Recognition, Academic Press: New York, 1990.
J.W. Tukey,Exploratory Data Analysis, Addison-Wesley: Reading, MA, 1960.
L.I. Perlovsky, “Multiple sensor fusion and neural networks,” DARPA Neural Network Study, MIT Lincoln Laboratory, Lexington, MA, 1987.
L.I. Perlovsky and M. M. McManus, “Maximum likelihood neural network for sensor fusion and adaptive classification,”Neural Networks, vol. 4, pp. 89–102, 1991.
A. Kaufmann,Introduction to the Theory of Fuzzy Sets, vol. I, Academic Press: New York, 1975.
L.I. Perlovsky, “Neural networks for sensor fusion and adaptive classification,” presented at 1st Annual International Neural Network Society Meeting, Boston, MA, 1988.
D.M. Titterington, A.F.M. Smith, and U.E. Makov,Statistical Analysis of Finite Mixture Distributions, John Wiley: New York 1985.
J. Jaskolski, “Clustering of the Fukunaga's standard data set using ART,” 1993, unpublished.
K. Fukunaga and R.R. Hayes, “Statistical classifier design and evaluation,” Purdue University, West Lafayette, IN, Report TR-EE 88-19, 1988.
L.I. Perlovsky and T.L. Marzetta, “Estimating a covariance matrix in a sensor fusion problem,”IEEE Trans. Signal Process., vol. 40, pp. 2097–2100, 1992.
H. Cramer,Mathematical Methods of Statistics, Princeton University Press: Princeton, NJ, 1963.
M.B. Priestley,Spectral Analysis and Time Series, Academic Press: New York, 1981.
L.I. Perlovsky, “Frequency estimates for simple oscillating systems under random forcing,” inLinear Algebra in Signals, Systems and Control, B. N. Datta C.R. Johnson, M.A. Kaashoek, R.J. Plemmons, and E.D. Sontag, eds., SIAM Press: Philadelphia PA, 1988.
L.I. Perlovsky, “Fuzzy tracking of multipleobjects,” presented at 1st IEEE Workshop on Neural Networks for Signal Processing, Princeton, NJ, 1991.
L.I. Perlovsky, and C.P. Plum, “Model based multiple target tracking using MLANS,” inProc. 3rd Biennial Acoustics, Speech, and Signal Processing Miniconference, Boston, MA, 1991.
S.S. Blackman,Multiple Target Tracking with Radar Applications, Artech: Norwood, MA, 1986.
R.A. Singer, R.G. Sea, and R.B. Housewright, “Derivation and evaluation of improved tracking filters for use in dense multitarget environments,”IEEE Trans. Inform. Theory, vol. IT-20, pp. 423–432, 1974.
Y. Bar-Shalom and E. Tse, “Tracking in a cluttered environment with probabilistic data association,”Automatica, vol. 11, pp. 451–460, 1975.
N. Chomsky,Knowledge of Language: Its Nature, Origin, and Use. Praeger: New York, 1986.
S. Watanabe,Pattern Recognition: Human and Mechanical, John Wiley: New York, 1985.
M. Siebert and A.M. Waxman, “Early vision applications of feature-map diffusion-enhancement nets,” presented at 1st Annual International Neural Network Society Meeting, Boston, MA, 1988.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Perlovsky, L.I. Computational concepts in classification: Neural networks, statistical pattern recognition, and model-based vision. J Math Imaging Vis 4, 81–110 (1994). https://doi.org/10.1007/BF01250006
Issue Date:
DOI: https://doi.org/10.1007/BF01250006