Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Numerical Algorithms
Published in: Numerical Algorithms 2/2020

24-03-2020 | Original Paper

Secant variable projection method for solving nonnegative separable least squares problems

Authors: Xiongfeng Song, Wei Xu, Ken Hayami, Ning Zheng

Published in: Numerical Algorithms | Issue 2/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The variable projection method is a classical and efficient method for solving separable nonlinear least squares (SNLLS) problems. However, it is hard to handle the constrained SNLLS problems since the explicit form of the Jacobian matrix is required in each iteration. In this paper, we propose a secant variable projection (SVP) method, which employs a rank-one update to estimate the Jacobian matrices. The main advantages of our method are efficiency and ease of applicability to constrained SNLLS problems. The local convergence of our SVP method is also analyzed. Finally, some data fitting and image processing problems are solved to compare the performance of our proposed method with the classical variable projection method. Numerical results illustrate the efficiency and stability of our proposed SVP method in solving the SNLLS problems arising from the blind deconvolution problems.
previous article Enhancement of the Kaczmarz algorithm with projection adjustment

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Berisha, S., Nagy, J.G.: Iterative methods for image restoration. In: Academic Press Library in Signal Processing, vol. 4, pp 193–247. Elsevier (2014)
2.
Chung, J., Haber, E., Nagy, J.: Numerical methods for coupled super-resolution. Inverse Prob. 22(4), 1261 (2006)MathSciNetMATH
3.
Chung, J., Nagy, J.G.: An efficient iterative approach for large-scale separable nonlinear inverse problems. SIAM J. Sci. Comput. 31(6), 4654–4674 (2010)MathSciNetMATH
4.
Chung, J., Nagy, J.G., O’Leary, D.P.: A weighted GCV method for Lanczos hybrid regularization. Electronic Transactions on Numerical Analysis, 28(Electronic Transactions on Numerical Analysis) (2008)
5.
Coleman, T.F., Xu, W.: Automatic differentiation in MATLAB using ADMAT with applications, vol. 27. SIAM (2016)
6.
Cornelio, A., Piccolomini, E.L., Nagy, J.G.: Constrained numerical optimization methods for blind deconvolution. Numerical Algorithms 65(1), 23–42 (2014)MathSciNetMATH
7.
Dennis, J.E. Jr., Schnabel, R.B.: Numerical methods for unconstrained optimization and nonlinear equations, vol. 16. SIAM (1996)
8.
Golub, G., Pereyra, V.: Separable nonlinear least squares: the variable projection method and its applications. Inverse Prob. 19(2), R1 (2003)MathSciNetMATH
9.
Golub, G.H., Pereyra, V.: The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10(2), 413–432 (1973)MathSciNetMATH
10.
Hansen, P.C.: Discrete inverse problems: insight and algorithms, vol. 7. SIAM (2010)
11.
Hansen, P.C., Nagy, J.G., O’leary, D.P.: Deblurring images: matrices, spectra, and filtering, vol. 3. SIAM (2006)
12.
Hansen, P.C., Pereyra, V., Scherer, G.: Least squares data fitting with applications. John Hopkins University Press, Baltimore (2013)MATH
13.
Kaufman, L.: A variable projection method for solving separable nonlinear least squares problems. BIT Numer. Math. 15(1), 49–57 (1975)MathSciNetMATH
14.
Kaufman, L., Pereyra, V.: A method for separable nonlinear least squares problems with separable nonlinear equality constraints. SIAM J. Numer. Anal. 15 (1), 12–20 (1978)MathSciNetMATH
15.
Kim, C.-T., Lee, J.-J.: Training two-layered feedforward networks with variable projection method. IEEE Trans. Neural Netw. 19(2), 371–375 (2008)
16.
Lawton, W.H., Sylvestre, E.A.: Elimination of linear parameters in nonlinear regression. Technometrics 13(3), 461–467 (1971)MATH
17.
Madsen, K., Nielsen, H.B., Tingleff, O.: Methods for nonlinear least squares problems. Informatics and Mathematical Modelling, Techincal University of Denmark (2004)
18.
Mullen, K.M., Van Stokkum, I.H.M.: The variable projection algorithm in time-resolved spectroscopy, microscopy and mass spectrometry applications. Numerical Algorithms 51(3), 319–340 (2009)MathSciNetMATH
19.
Nagy, J.G., Palmer, K., Perrone, L.: Iterative methods for image deblurring: a Matlab object-oriented approach. Numerical Algorithms 36(1), 73–93 (2004)MathSciNetMATH
20.
Nagy, J.G., Ruthotto, L.: Lap: a linearize and project method for solving inverse problems with coupled variables. Imaging (MRI) 1, 6 (2018)MATH
21.
O’leary, D.P., Rust, B.W.: Variable projection for nonlinear least squares problems. Comput. Optim. Appl. 54(3), 579–593 (2013)MathSciNetMATH
22.
Ruhe, A., Wedin, P.A.: Algorithms for separable nonlinear least squares problems. Siam Rev. 22(3), 318–337 (1980)MathSciNetMATH
23.
Sima, D.M., Van Huffel, S.: Separable nonlinear least squares fitting with linear bound constraints and its application in magnetic resonance spectroscopy data quantification. Elsevier Science Publishers B. V. (2007)
24.
Wold, H., Lyttkens, E.: Nonlinear iterative partial least squares (NIPALS) estimation procedures. Bull. Int. Stat. Inst. 43, 29–51 (1969)MATH
25.
Wright, S., Nocedal, J.: Numerical optimization. Springer Science 35(67-68), 7 (1999)MATH
26.
Xu, W., Coleman, T.F., Liu, G.: A secant method for nonlinear least-squares minimization. Comput. Optim. Appl. 51(1), 159–173 (2012)MathSciNetMATH
27.
Zheng, N., Hayami, K., Yin, J.-F.: Modulus-type inner outer iteration methods for nonnegative constrained least squares problems. SIAM J. Matrix Anal. Appl. 37(3), 1250–1278 (2016)MathSciNetMATH
Metadata
Title
Secant variable projection method for solving nonnegative separable least squares problems
Authors
Xiongfeng Song
Wei Xu
Ken Hayami
Ning Zheng
Publication date
24-03-2020
Publisher
Springer US
Published in
Numerical Algorithms / Issue 2/2020
Print ISSN: 1017-1398
Electronic ISSN: 1572-9265
DOI
https://doi.org/10.1007/s11075-019-00835-2

Other articles of this Issue 2/2020

Numerical Algorithms 2/2020 Go to the issue

Original Paper

An incremental bundle method for portfolio selection problem under second-order stochastic dominance

Original Paper

Multiscale radial kernels with high-order generalized Strang-Fix conditions

Original Paper

Analysis and application of the interpolating element-free Galerkin method for extended Fisher–Kolmogorov equation which arises in brain tumor dynamics modeling

Original Paper

Efficient methods for nonlinear time fractional diffusion-wave equations and their fast implementations

Original Paper

Orthogonal sequences constructed from quasi-orthogonal ultraspherical polynomials

Original Paper

An interior penalty approach to a large-scale discretized obstacle problem with nonlinear constraints

Premium Partner

    Image Credits