Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Information Theory, Machine Learning, and Reproducing Kernel Hilbert Spaces Read chapter Read first chapter

2010 | OriginalPaper | Chapter

11. Correntropy for Random Processes: Properties and Applications in Signal Processing

Authors : Puskal Pokharel, Ignacio Santamaria, Jianwu Xu, Kyu-hwa Jeong, Weifeng Liu

Published in: Information Theoretic Learning

Publisher: Springer New York

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The previous chapter defined cross-correntropy for the case of a pair of scalar random variables, and presented applications in statistical inference. This chapter extends the definition of correntropy for the case of random (or stochastic) processes, which are index sets of random variables. In statistical signal processing the index set is time; we are interested in random variables that are a function of time and the goal is to quantify their statistical dependencies (although the index set can also be defined over inputs or channels of multivariate random variables). The autocorrelation function, which measures the statistical dependency between random variables at two different times, is conventionally utilized for this goal. Hence, we generalize the definition of autocorrelation to an autocorrentropy function. The name correntropywas coined to reflect the fact that the function “looks like” correlation but the sum over the lags (or over dimensions of the multivariate random variable) is the information potential (i.e., the argument of Renyi’s quadratic entropy). The definition of cross-correntropy for random variables carries over to time series with a minor but important change in the domain of the variables that now are an index set of lags. When it is clear from the context, we simplify the terminology and refer to the different functions (autocorrentropy, or crosscorrentropy) simply as correntropy function, but keep the word “function” to distinguish them from Chapter 10 quantities.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Correntropy for Random Variables: Properties and Applications in Statistical Inference
Literature
5.
Amari S., Cichocki A., Yang H., A new learning algorithm for blind signal separation. Advances in Neural Information Processing Systems, vol. 8 pp. 757–763, MIT Press, Cambridge, MA, 1996.
8.
15.
22.
Beadle E., Schroeder J., Moran B., An overview of Renyi’s entropy and some potential applications, 42 nd Asilomar Conference on Signals, Systems and Computers, October 2008.
29.
Belouchrani A., Abed-Meraim K., Cardoso J., Moulines E., A blind source separation technique using second-order statistics, IEEE Trans. Signal Process. 45(2):434–444, 1997.CrossRef
34.
Bercher J., Vignat C., Estimating the Entropy of a Signal with Applications, IEEE Trans. Signal Process., 48(6):1687–1694, 2000.CrossRef
42.
Brown J., Puckette M., Calculation of a “narrowed” autocorrelation function, J. Acoust. Soc. Am., 85:1595–1601, 1989.CrossRef
46.
Cardoso J., Souloumiac A., Blind beamforming for non-Gaussian signals, Radar Signal Process., IEE Proc. F, 140(6):362–370, December 1993.CrossRef
62.
Comon P., Independent component analysis, a new concept?, Signal Process., 36(3):287–314, 1994.CrossRefMATH
123.
Gretton, A., Herbrich R., Smola A., Bousquet O., Schölkopf B., Kernel Methods for Measuring Independence,” J. Mach. Learn. Res., 6:2075–2129, 2005.MATHMathSciNet
134.
Hardoon D., Szedmak S., Shawe-Taylor J., Canonical correlation analysis: an overview with application to learning methods, Neur. Comput., 16(12):2664–2699, Dec. 2004.CrossRef
147.
Hess W., Pitch Determination of Speech Signals. Springer, New York, 1993.
150.
Hild II K., Erdogmus D., Principe J., An analysis of entropy estimators for blind source separation, Signal Process., 86(1):182–194, 2006.CrossRefMATH
155.
Hyvarinen A., Fast and Robust Fixed-Point Algorithms for Independent Component Analysis, IEEE Trans. Neural Netw., 10(3):626–634, 1999.CrossRef
166.
Jeong K.H., Liu W., Principe J., The correntropy MACE filter, Pattern Recogn., 42(5):871–885, 2009.CrossRefMATH
167.
Jeong K.W., Principe J., Enhancing the correntropy MACE filter with random projections, Neurocomputing, 72(1–3):102–111, 2008.CrossRef
172.
Jutten C., Herault J., Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture. Signal Process., 24:1–10, 1991.CrossRefMATH
189.
Kumar B., Minimum variance synthetic discriminant functions, J. Opt. Soc. Am., A3(10):1579–1584, 1986.CrossRef
190.
Kumar B., Tutorial survey of composite filter designs for optical correlators, Appl. Opt., 31:4773–4801, 1992.CrossRef
191.
Kumar B., Savvides M., Xie C., Venkataramani K., Biometric verification with correlation filters, Appl. Opt., 43(2):391–402, 2004.CrossRef
198.
Li R., Liu W., Principe J., A unifying criterion for blind source separation based on correntropy, Signal Process., Special Issue on ICA, 8(78):1872–1881.
201.
Liu W., Pokharel P., Principe J., Correntropy: Properties and applications in non Gaussian signal processing, IEEE Trans. Sig. Proc., 55(11):5286–5298, 2007.CrossRefMathSciNet
203.
Loève, M.M., Probability Theory, VanNostrand, Princeton, NJ, 1955.MATH
209.
Mahalanobis A., Kumar B., Casasent D., Minimum average correlation energy filters, Appl. Opt., 26(17):3633–3640, 1987.CrossRef
210.
Mahalanobis A., Forman A., Bower M., Cherry R., Day N., Multi-class SAR ATR using shift invariant correlation filters, Pattern Recogn., 27:619–626 Special Issue on Correlation Filters and Neural Networks, 1994.
211.
Mardia K., Jupp P., Directional Statistics, Wiley, New York, 2000.MATH
217.
Mercer J., Functions of positive and negative type, and their connection with the theory of integral equations, Philosoph. Trans. Roy. Soc. Lond., 209:415–446, 1909.CrossRefMATH
219.
Mika S., Ratsch G., Weston J., Scholkopf B., Muller K., Fisher discriminant analysis with kernels. In Proceedings of IEEE International Workshop on Neural Networks for Signal Processing, pages 41–48, Madison, USA, August 23–25, 1999.
235.
Papoulis A., Probability, Random Variables and Stochastic Processes, McGraw-Hill, New York, 1965.MATH
238.
Parzen E., Statistical inference on time series by Hilbert space methods, Tech. Report 23, Stat. Dept., Stanford Univ., 1959.
242.
Patterson R., Holdsworth J., Nimmo-Smith I., Rice P., SVOS final report, Part B: Implementing a gammatone filterbank, Appl. Psychol. Unit Rep.2341, 1988
248.
Pokharel P., Xu J., Erdogmus D., Principe J., A closed form solution for a nonlinear Wiener filter, Proc. IEEE Int. Conf. Acoustics Speech and Signal Processing, Toulose, France, 2006.
253.
Principe J., Euliano N., Lefebvre C., Neural Systems: Fundamentals through Simulations, CD-ROM textbook, John Wiley, New York, 2000.
273.
Ross T., Worrell S., Velten V., Mossing J., Bryant M., Standard SAR ATR evaluation experiments using the MSTAR public release data set, in: Proceedings of the SPIE, vol. 3370, 1998, pp. 566–573.
283.
Santos J., Alexandre L., Sa J., The error entropy minimization algorithm for neural network classification, in A. Lofti (Ed.), Int Conf. Recent Advances in Soft Computing, pp. 92–97, 2004.
288.
Schölkopf, B., Burges, C.J.C., Smola, A.J. (Eds.), Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge, MA, 1999.
289.
Schölkopf B. and Smola A., Learning with Kernels. MIT Press, Cambridge, MA, 2002
294.
Shawe-Taylor J. Cristianini N., Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge, UK, 2004.CrossRef
300.
Silverman B., Density Estimation for Statistics and Data Analysis, Chapman and Hall, London, 1986.CrossRefMATH
323.
329.
Wang D., Brown G., Computational Auditory Scene Analysis—Principles, Algorithms, and Applications. Wiley, New York, 2006.
335.
Wu H., Principe J., Simultaneous diagonalization in the frequency domain for source separation, Proc. First Int. Workshop on Ind. Comp. Anal. ICA’99, 245–250, Aussois, France, 1999.
338.
Xie C., Savvides M., Kumar B., Kernel correlation filter based redundant class dependence feature analysis on FRGC2.0 data, in: Proc. second Int. Workshop Analysis Modeling Faces Gesture (AMFG), Beijing, 2005.
342.
Xu J., Principe J., A pitch detector based on a generalized correlation function, IEEE Trans. Audio, Speech Lang. Process., 16(8):1420–1432, 2008.CrossRef
343.
Xu J., Nonlinear Signal Processing Based on Reproducing Kernel Hilbert Space, Ph.D. Thesis, University of Florida, Gainesville, 2008.
Metadata
Title
Correntropy for Random Processes: Properties and Applications in Signal Processing
Authors
Puskal Pokharel
Ignacio Santamaria
Jianwu Xu
Kyu-hwa Jeong
Weifeng Liu
Copyright Year
2010
Publisher
Springer New York
Book
Information Theoretic Learning

Print ISBN: 978-1-4419-1569-6

Electronic ISBN: 978-1-4419-1570-2

Copyright Year: 2010

DOI
https://doi.org/10.1007/978-1-4419-1570-2_11

Premium Partner

    Image Credits