nach oben

Numerical Algorithms

Erschienen in:

24.03.2020 | Original Paper

Secant variable projection method for solving nonnegative separable least squares problems

verfasst von: Xiongfeng Song, Wei Xu, Ken Hayami, Ning Zheng

Erschienen in: Numerical Algorithms | Ausgabe 2/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The variable projection method is a classical and efficient method for solving separable nonlinear least squares (SNLLS) problems. However, it is hard to handle the constrained SNLLS problems since the explicit form of the Jacobian matrix is required in each iteration. In this paper, we propose a secant variable projection (SVP) method, which employs a rank-one update to estimate the Jacobian matrices. The main advantages of our method are efficiency and ease of applicability to constrained SNLLS problems. The local convergence of our SVP method is also analyzed. Finally, some data fitting and image processing problems are solved to compare the performance of our proposed method with the classical variable projection method. Numerical results illustrate the efficiency and stability of our proposed SVP method in solving the SNLLS problems arising from the blind deconvolution problems.

Vorheriger Artikel Enhancement of the Kaczmarz algorithm with projection adjustment

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Berisha, S., Nagy, J.G.: Iterative methods for image restoration. In: Academic Press Library in Signal Processing, vol. 4, pp 193–247. Elsevier (2014)

Chung, J., Haber, E., Nagy, J.: Numerical methods for coupled super-resolution. Inverse Prob. 22(4), 1261 (2006)MathSciNetMATH

Chung, J., Nagy, J.G.: An efficient iterative approach for large-scale separable nonlinear inverse problems. SIAM J. Sci. Comput. 31(6), 4654–4674 (2010)MathSciNetMATH

Chung, J., Nagy, J.G., O’Leary, D.P.: A weighted GCV method for Lanczos hybrid regularization. Electronic Transactions on Numerical Analysis, 28(Electronic Transactions on Numerical Analysis) (2008)

Coleman, T.F., Xu, W.: Automatic differentiation in MATLAB using ADMAT with applications, vol. 27. SIAM (2016)

Cornelio, A., Piccolomini, E.L., Nagy, J.G.: Constrained numerical optimization methods for blind deconvolution. Numerical Algorithms 65(1), 23–42 (2014)MathSciNetMATH

Dennis, J.E. Jr., Schnabel, R.B.: Numerical methods for unconstrained optimization and nonlinear equations, vol. 16. SIAM (1996)

Golub, G., Pereyra, V.: Separable nonlinear least squares: the variable projection method and its applications. Inverse Prob. 19(2), R1 (2003)MathSciNetMATH

Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10(2), 413–432 (1973)MathSciNetMATH

10.

Hansen, P.C.: Discrete inverse problems: insight and algorithms, vol. 7. SIAM (2010)

11.

Hansen, P.C., Nagy, J.G., O’leary, D.P.: Deblurring images: matrices, spectra, and filtering, vol. 3. SIAM (2006)

12.

Hansen, P.C., Pereyra, V., Scherer, G.: Least squares data fitting with applications. John Hopkins University Press, Baltimore (2013)MATH

13.

Kaufman, L.: A variable projection method for solving separable nonlinear least squares problems. BIT Numer. Math. 15(1), 49–57 (1975)MathSciNetMATH

14.

Kaufman, L., Pereyra, V.: A method for separable nonlinear least squares problems with separable nonlinear equality constraints. SIAM J. Numer. Anal. 15 (1), 12–20 (1978)MathSciNetMATH

15.

Kim, C.-T., Lee, J.-J.: Training two-layered feedforward networks with variable projection method. IEEE Trans. Neural Netw. 19(2), 371–375 (2008)

16.

Lawton, W.H., Sylvestre, E.A.: Elimination of linear parameters in nonlinear regression. Technometrics 13(3), 461–467 (1971)MATH

17.

Madsen, K., Nielsen, H.B., Tingleff, O.: Methods for nonlinear least squares problems. Informatics and Mathematical Modelling, Techincal University of Denmark (2004)

18.

Mullen, K.M., Van Stokkum, I.H.M.: The variable projection algorithm in time-resolved spectroscopy, microscopy and mass spectrometry applications. Numerical Algorithms 51(3), 319–340 (2009)MathSciNetMATH

19.

Nagy, J.G., Palmer, K., Perrone, L.: Iterative methods for image deblurring: a Matlab object-oriented approach. Numerical Algorithms 36(1), 73–93 (2004)MathSciNetMATH

20.

Nagy, J.G., Ruthotto, L.: Lap: a linearize and project method for solving inverse problems with coupled variables. Imaging (MRI) 1, 6 (2018)MATH

21.

O’leary, D.P., Rust, B.W.: Variable projection for nonlinear least squares problems. Comput. Optim. Appl. 54(3), 579–593 (2013)MathSciNetMATH

22.

Ruhe, A., Wedin, P.A.: Algorithms for separable nonlinear least squares problems. Siam Rev. 22(3), 318–337 (1980)MathSciNetMATH

23.

Sima, D.M., Van Huffel, S.: Separable nonlinear least squares fitting with linear bound constraints and its application in magnetic resonance spectroscopy data quantification. Elsevier Science Publishers B. V. (2007)

24.

Wold, H., Lyttkens, E.: Nonlinear iterative partial least squares (NIPALS) estimation procedures. Bull. Int. Stat. Inst. 43, 29–51 (1969)MATH

25.

Wright, S., Nocedal, J.: Numerical optimization. Springer Science 35(67-68), 7 (1999)MATH

26.

Xu, W., Coleman, T.F., Liu, G.: A secant method for nonlinear least-squares minimization. Comput. Optim. Appl. 51(1), 159–173 (2012)MathSciNetMATH

27.

Zheng, N., Hayami, K., Yin, J.-F.: Modulus-type inner outer iteration methods for nonnegative constrained least squares problems. SIAM J. Matrix Anal. Appl. 37(3), 1250–1278 (2016)MathSciNetMATH

Titel: Secant variable projection method for solving nonnegative separable least squares problems
verfasst von: Xiongfeng Song
Wei Xu
Ken Hayami
Ning Zheng
Publikationsdatum: 24.03.2020
Verlag: Springer US
Erschienen in: Numerical Algorithms / Ausgabe 2/2020
Print ISSN: 1017-1398
Elektronische ISSN: 1572-9265
DOI: https://doi.org/10.1007/s11075-019-00835-2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 2/2020

An interior penalty approach to a large-scale discretized obstacle problem with nonlinear constraints

Efficient methods for nonlinear time fractional diffusion-wave equations and their fast implementations

Superconvergence analysis of a two-grid method for an energy-stable Ciarlet-Raviart type scheme of Cahn-Hilliard equation

On a family of non-oscillatory subdivision schemes having regularity Cr with r > 1

An efficient third-order scheme for BSDEs based on nonequidistant difference scheme

Multiscale radial kernels with high-order generalized Strang-Fix conditions