Abstract
In this note we show that the kernel PCA algorithm of Schölkopf, Smola, and Müller (Neural Computation, 10, 1299–1319.) can be interpreted as a form of metric multidimensional scaling (MDS) when the kernel function k(x, y) is isotropic, i.e. it depends only on ‖x − y‖. This leads to a metric MDS algorithm where the desired configuration of points is found via the solution of an eigenproblem rather than through the iterative optimization of the stress objective function. The question of kernel choice is also discussed.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Berg, C., Christensen, J. P. R., & Ressel, P. (1984). Harmonic analysis on semigroups New York, NY: Springer-Verlag.
Cox, T. F. & Cox, M. A. A. (1994). Multidimensional scaling. London: Chapman and Hall.
Critchley, F. (1978). Multidimensionsal scaling: A short critique and a new method. In L. C. A. Corsten & J. Hermnas (Eds.), COMPSTAT 1978. Vienna: Physica-Verlag.
Kruskal, J. B. & Wish, M. (1978). Multidimensional scaling. Beverly Hills: Sage Publications.
Mardia, K. V., Kent, J. T., & Bibby, J. M. (1979). Multivariate analysis. London: Academic Press.
Sammon, J.W. (1969). A nonlinear mapping for data structure analysis. IEEE Trans. on Computers, 18, 401-409.
Schölkopf, B., Smola, A., & Müller, K.-R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10, 1299-1319.
Vapnik, V. N. (1995). The nature of statistical learning theory. New York: Springer Verlag.
Wahba, G. (1990). Spline models for observational data. Philadelphia, PA: Society for Industrial and Applied Mathematics. CBMS-NSF Regional Conference series in applied mathematics.
Williams, C. K. I. & Barber, D. (1998). Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20:12, 1342-1351.
Yaglom, A. M. (1987). Correlation theory of stationary and related random functions vol. I:Basic results. Berlin: Springer Verlag.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Williams, C.K. On a Connection between Kernel PCA and Metric Multidimensional Scaling. Machine Learning 46, 11–19 (2002). https://doi.org/10.1023/A:1012485807823
Issue Date:
DOI: https://doi.org/10.1023/A:1012485807823