Skip to main content
Top
Published in: Soft Computing 2/2020

13-04-2019 | Methodologies and Application

Application of a new accelerated algorithm to regression problems

Authors: Avinash Dixit, D. R. Sahu, Amit Kumar Singh, T. Som

Published in: Soft Computing | Issue 2/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Many iterative algorithms like Picard, Mann, Ishikawa are very useful to solve fixed point problems of nonlinear operators in real Hilbert spaces. The recent trend is to enhance their convergence rate abruptly by using inertial terms. The purpose of this paper is to investigate a new inertial iterative algorithm for finding the fixed points of nonexpansive operators in the framework of Hilbert spaces. We study the weak convergence of the proposed algorithm under mild assumptions. We apply our algorithm to design a new accelerated proximal gradient method. This new proximal gradient technique is applied to regression problems. Numerical experiments have been conducted for regression problems with several publicly available high-dimensional datasets and compare the proposed algorithm with already existing algorithms on the basis of their performance for accuracy and objective function values. Results show that the performance of our proposed algorithm overreaches the other algorithms, while keeping the iteration parameters unchanged.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
go back to reference Acedo GL, Xu HK (2007) Iterative methods for strict pseudo-contractions in Hilbert spaces. Nonlinear Anal: Theory Methods Appl 67(7):2258–2271MathSciNetCrossRef Acedo GL, Xu HK (2007) Iterative methods for strict pseudo-contractions in Hilbert spaces. Nonlinear Anal: Theory Methods Appl 67(7):2258–2271MathSciNetCrossRef
go back to reference Agarwal RP, O’Regan D, Sahu DR (2007) Iterative construction of fixed points of nearly asymptotically nonexpansive mappings. J Nonlinear Convex Anal 8(1):61–79MathSciNetMATH Agarwal RP, O’Regan D, Sahu DR (2007) Iterative construction of fixed points of nearly asymptotically nonexpansive mappings. J Nonlinear Convex Anal 8(1):61–79MathSciNetMATH
go back to reference Alvarez F (2004) Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space. SIAM J optim 14:773–782MathSciNetCrossRef Alvarez F (2004) Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space. SIAM J optim 14:773–782MathSciNetCrossRef
go back to reference Alvarez F, Attouch H (2001) An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal 9:3–11MathSciNetCrossRef Alvarez F, Attouch H (2001) An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal 9:3–11MathSciNetCrossRef
go back to reference Bauschke HH, Borwein JM (1996) On projection algorithms for solving convex feasibility problems. SIAM Rev 38:367–426MathSciNetCrossRef Bauschke HH, Borwein JM (1996) On projection algorithms for solving convex feasibility problems. SIAM Rev 38:367–426MathSciNetCrossRef
go back to reference Bauschke HH, Combttes PL (2011) Convex Analysis and Monotone Operator Theory in Hilbert space. Springer, BerlinCrossRef Bauschke HH, Combttes PL (2011) Convex Analysis and Monotone Operator Theory in Hilbert space. Springer, BerlinCrossRef
go back to reference Beck A, Teboulle M (2009) Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans Image Process 18(11):2419–2434MathSciNetCrossRef Beck A, Teboulle M (2009) Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans Image Process 18(11):2419–2434MathSciNetCrossRef
go back to reference Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2(1):183–202MathSciNetCrossRef Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2(1):183–202MathSciNetCrossRef
go back to reference Bot RI, Csetnek ER (2016) An inertial alternating direction method of multipliers. Minimax Theory Appl 1:29–49MathSciNetMATH Bot RI, Csetnek ER (2016) An inertial alternating direction method of multipliers. Minimax Theory Appl 1:29–49MathSciNetMATH
go back to reference Bot RI, Csetnek ER, Hendrich C (2015) Inertial Douglas–Rachford splitting for monotone inclusion problems. Appl Math Comput 256:472–487MathSciNetMATH Bot RI, Csetnek ER, Hendrich C (2015) Inertial Douglas–Rachford splitting for monotone inclusion problems. Appl Math Comput 256:472–487MathSciNetMATH
go back to reference Byrne CL (2004) A unified treatment of some iterative algorithm in signal processing and image reconstruction. Inverse Probl 18:441–453MathSciNetCrossRef Byrne CL (2004) A unified treatment of some iterative algorithm in signal processing and image reconstruction. Inverse Probl 18:441–453MathSciNetCrossRef
go back to reference Chambolle A, Dossal C (2015) Dossal On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. J Optim Theory Appl 166:968–982MathSciNetCrossRef Chambolle A, Dossal C (2015) Dossal On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. J Optim Theory Appl 166:968–982MathSciNetCrossRef
go back to reference Chang SS, Wang G, Wang L, Tang YK, Ma ZL (2014) \(\bigtriangleup \)-convergence theorems for multi-valued nonexpansive mapping in hyperbolic spaces. Appl Math Comput 249:535–540MathSciNetMATH Chang SS, Wang G, Wang L, Tang YK, Ma ZL (2014) \(\bigtriangleup \)-convergence theorems for multi-valued nonexpansive mapping in hyperbolic spaces. Appl Math Comput 249:535–540MathSciNetMATH
go back to reference Chen C, Ma S, Yang J (2014) A general inertial proximal point algorithm for mixed variational inequality problem. SIAM J Optim 25(4):2120–2142MathSciNetCrossRef Chen C, Ma S, Yang J (2014) A general inertial proximal point algorithm for mixed variational inequality problem. SIAM J Optim 25(4):2120–2142MathSciNetCrossRef
go back to reference Chidume CE, Mutangadura S (2001) An example on the Mann iteration method for Lipschitzian pseudocontractions. Proc Am Math Soc 129:2359–2363CrossRef Chidume CE, Mutangadura S (2001) An example on the Mann iteration method for Lipschitzian pseudocontractions. Proc Am Math Soc 129:2359–2363CrossRef
go back to reference Franklin J (1980) Methods of mathematical economics. Springer Verlag, New YorkCrossRef Franklin J (1980) Methods of mathematical economics. Springer Verlag, New YorkCrossRef
go back to reference Lions JL, Stampacchia G (1967) Variational inequalities. Commun Pure Appl Math 20:493–519CrossRef Lions JL, Stampacchia G (1967) Variational inequalities. Commun Pure Appl Math 20:493–519CrossRef
go back to reference Lorenz DA, Pock T (2015) An inertial forword–backword algorithm for monotone inclusions. J Math Imaging Vis 51:311–325CrossRef Lorenz DA, Pock T (2015) An inertial forword–backword algorithm for monotone inclusions. J Math Imaging Vis 51:311–325CrossRef
go back to reference Mercier B (1980) Mechanics and Variational Inequalities, Lecture Notes. Orsay Centre of Paris University Mercier B (1980) Mechanics and Variational Inequalities, Lecture Notes. Orsay Centre of Paris University
go back to reference Nesterov YE (2007) Gradient methods for minimizing composite objective functions, Technical report, Center for Operations Research and Econometrics (CORE), Catholie University of Louvain Nesterov YE (2007) Gradient methods for minimizing composite objective functions, Technical report, Center for Operations Research and Econometrics (CORE), Catholie University of Louvain
go back to reference Nesterov YE (1983) A method for unconstrained convex minimization problem with the rate of convergence \(O(\frac{1}{k^2})\). Dokl Akad Nauk SSSR 269(3):543–547MathSciNet Nesterov YE (1983) A method for unconstrained convex minimization problem with the rate of convergence \(O(\frac{1}{k^2})\). Dokl Akad Nauk SSSR 269(3):543–547MathSciNet
go back to reference Nesterov Y (2004) Introductory lectures on convex optimization: a basic course, Applied Optimization, vol 87. Kluwer Academic Publishers, BostonCrossRef Nesterov Y (2004) Introductory lectures on convex optimization: a basic course, Applied Optimization, vol 87. Kluwer Academic Publishers, BostonCrossRef
go back to reference Ochs P, Chen Y, Brox T, Pock T (2014) Inertial proximal algorithm for non-convex optimization. SIAM J Imaging Sci 7(2):1388–1419MathSciNetCrossRef Ochs P, Chen Y, Brox T, Pock T (2014) Inertial proximal algorithm for non-convex optimization. SIAM J Imaging Sci 7(2):1388–1419MathSciNetCrossRef
go back to reference Polyak BT (1964) Some methods of speeding up the convergence od iteration methods. U S S R Comput Math Math Phys 4:1–17CrossRef Polyak BT (1964) Some methods of speeding up the convergence od iteration methods. U S S R Comput Math Math Phys 4:1–17CrossRef
go back to reference Sahu DR (2011) Applications of the S-iteration process to constrained minimization problems and split feasibility problems. Fixed Point Theory Appl 12:187–204MathSciNetMATH Sahu DR (2011) Applications of the S-iteration process to constrained minimization problems and split feasibility problems. Fixed Point Theory Appl 12:187–204MathSciNetMATH
go back to reference Sakurai K, Liduka H (2014) Acceleration of the Halpern algorithm to search for a fixed point of a nonexpansive mapping. Fixed Point Theory Appl 202 Sakurai K, Liduka H (2014) Acceleration of the Halpern algorithm to search for a fixed point of a nonexpansive mapping. Fixed Point Theory Appl 202
go back to reference Suparatulatorn R, Cholamjiak P (2016) The modified S-iteration process for nonexpansive mappings in CAT(k) spaces. Fixed Point Theory Appl 25 Suparatulatorn R, Cholamjiak P (2016) The modified S-iteration process for nonexpansive mappings in CAT(k) spaces. Fixed Point Theory Appl 25
go back to reference Takahashi W, Takeuchi Y, Kubota R (2008) Strong convergence theorems by hybrid methods for families of nonexpansive mappings in Hilbert spaces. J Math Anal Appl 341(1):276–286MathSciNetCrossRef Takahashi W, Takeuchi Y, Kubota R (2008) Strong convergence theorems by hybrid methods for families of nonexpansive mappings in Hilbert spaces. J Math Anal Appl 341(1):276–286MathSciNetCrossRef
go back to reference Tibshirani R (1996) Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B 58(1):267–288MathSciNetMATH Tibshirani R (1996) Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B 58(1):267–288MathSciNetMATH
go back to reference Tseng P (2008) On accelerated proximal gradient methods for convex-concave optimization, Technical report, University of Washington, Seattle Tseng P (2008) On accelerated proximal gradient methods for convex-concave optimization, Technical report, University of Washington, Seattle
go back to reference Yao Y, Cho YJ, Liou YC (2011) Algorithms of common solutions of variational inclusions, mixed equilibrium problems and fixed point problems. Eur J Oper Res 212(2):242–250MathSciNetCrossRef Yao Y, Cho YJ, Liou YC (2011) Algorithms of common solutions of variational inclusions, mixed equilibrium problems and fixed point problems. Eur J Oper Res 212(2):242–250MathSciNetCrossRef
go back to reference Zhao LC, Chang SS, Kim JK (2013) Mixed type iteration for total asymptotically nonexpansive mappings in hyperbolic spaces. Fixed Points Theory Appl 353 Zhao LC, Chang SS, Kim JK (2013) Mixed type iteration for total asymptotically nonexpansive mappings in hyperbolic spaces. Fixed Points Theory Appl 353
Metadata
Title
Application of a new accelerated algorithm to regression problems
Authors
Avinash Dixit
D. R. Sahu
Amit Kumar Singh
T. Som
Publication date
13-04-2019
Publisher
Springer Berlin Heidelberg
Published in
Soft Computing / Issue 2/2020
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-019-03984-7

Other articles of this Issue 2/2020

Soft Computing 2/2020 Go to the issue

Premium Partner