Abstract
We study optimization methods for solving the maximum likelihood formulation of independent component analysis (ICA). We consider both the problem constrained to white signals and the unconstrained problem. The Hessian of the objective function is costly to compute, which renders Newton’s method impractical for large data sets. Many algorithms proposed in the literature can be rewritten as quasi-Newton methods, for which the Hessian approximation is cheap to compute. These algorithms are very fast on simulated data where the linear mixture assumption really holds. However, on real signals, we observe that their rate of convergence can be severely impaired. In this paper, we investigate the origins of this behavior, and show that the recently proposed Preconditioned ICA for Real Data (Picard) algorithm overcomes this issue on both constrained and unconstrained problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Comon, P.: Independent component analysis, a new concept? Sig. Process. 36(3), 287–314 (1994)
Himberg, J., Hyvärinen, A., Esposito, F.: Validating the independent components of neuroimaging time series via clustering and visualization. NeuroImage 22(3), 1214–1222 (2004)
Bell, A.J., Sejnowski, T.J.: An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7(6), 1129–1159 (1995)
Cardoso, J.-F.: Infomax and maximum likelihood for blind source separation. IEEE Sig. Process. Lett. 4(4), 112–114 (1997)
Zibulevsky, M.: Blind source separation with relative Newton method. In: Proceedings of the ICA, vol. 2003, pp. 897–902 (2003)
Choi, H., Choi, S.: A relative trust-region algorithm for independent component analysis. Neurocomputing 70(7), 1502–1510 (2007)
Palmer, J.A., Kreutz-Delgado, K., Makeig, S.: AMICA: an adaptive mixture of independent component analyzers with shared components. Technical report (2012)
Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626–634 (1999)
Hyvärinen, A.: The fixed-point algorithm and maximum likelihood estimation for independent component analysis. Neural Process. Lett. 10(1), 1–5 (1999)
Ablin, P., Cardoso, J.-F., Gramfort, A.: Faster ICA under orthogonal constraint. In: Proceedings of the IEEE ICASSP (2018)
Ablin, P., Cardoso, J.-F., Gramfort, A.: Faster independent component analysis by preconditioning with hessian approximations, Arxiv Preprint (2017)
Pham, D.T., Garat, P.: Blind separation of mixture of independent sources through a quasi-maximum likelihood approach. IEEE Trans. Sig. Process. 45(7), 1712–1725 (1997)
Cardoso, J.-F., Laheld, B.H.: Equivariant adaptive source separation. IEEE Trans. Sig. Process. 44(12), 3017–3030 (1996)
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)
Karimi, H., Nutini, J., Schmidt, M.: Linear convergence of gradient and proximal-gradient methods under the Polyak-Łojasiewicz condition. In: Frasconi, P., Landwehr, N., Manco, G., Vreeken, J. (eds.) ECML PKDD 2016. LNCS (LNAI), vol. 9851, pp. 795–811. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46128-1_50
Nocedal, J.: Updating Quasi-Newton matrices with limited storage. Math. Comput. 35(151), 773–782 (1980)
Delorme, A., Palmer, J., Onton, J., Oostenveld, R., Makeig, S.: Independent EEG sources are dipolar. PLoS ONE 7(2), e30135 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Ablin, P., Cardoso, JF., Gramfort, A. (2018). Accelerating Likelihood Optimization for ICA on Real Signals. In: Deville, Y., Gannot, S., Mason, R., Plumbley, M., Ward, D. (eds) Latent Variable Analysis and Signal Separation. LVA/ICA 2018. Lecture Notes in Computer Science(), vol 10891. Springer, Cham. https://doi.org/10.1007/978-3-319-93764-9_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-93764-9_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93763-2
Online ISBN: 978-3-319-93764-9
eBook Packages: Computer ScienceComputer Science (R0)