Abstract
We propose a bundle method for minimizing nonsmooth convex functions that combines both the level and the proximal stabilizations. Most bundle algorithms use a cutting-plane model of the objective function to formulate a subproblem whose solution gives the next iterate. Proximal bundle methods employ the model in the objective function of the subproblem, while level methods put the model in the subproblem’s constraints. The proposed algorithm defines new iterates by solving a subproblem that employs the model in both the objective function and in the constraints. One advantage when compared to the proximal approach is that the level set constraint provides a certain Lagrange multiplier, which is used to update the proximal parameter in a novel manner. We also show that in the case of inexact function and subgradient evaluations, no additional procedure needs to be performed by our variant to deal with inexactness (as opposed to the proximal bundle methods that require special modifications). Numerical experiments on almost 1,000 instances of different types of problems are presented. Our experiments show that the doubly stabilized bundle method inherits useful features of the level and the proximal versions, and compares favorably to both of them.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Astorino, A., Frangioni, A., Gaudioso, M., Gorgone, E.: Piecewise-quadratic approximations in convex numerical optimization. SIAM J. Optim. 21(4), 1418–1438 (2011). doi:10.1137/100817930
Bello-Cruz, J.Y., de Oliveria, W.: Level bundle-like algorithms for convex optimization. J. Glob. Optim. 59(4), 787–809 (2014). doi:10.1007/s10898-013-0096-4
Ben-Tal, A., Nemirovski, A.: Non-euclidean restricted memory level method for large-scale convex optimization. Math. Program. 102, 407–456 (2005). doi:10.1007/s10107-004-0553-4. http://dl.acm.org/citation.cfm?id=1057781.1057789
Bonnans, J., Gilbert, J., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization. Theoretical and Practical Aspects. Universitext. Springer, Berlin (2006). Second edition, xiv+490 pp
Brannlund, U., Kiwiel, K.C., Lindberg, P.O.: A descent proximal level bundle method for convex nondifferentiable optimization. Oper. Res. Lett. 17(3), 121–126 (1995). doi:10.1016/0167-6377(94)00056-C. http://www.sciencedirect.com/science/article/pii/016763779400056C
Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(2), 261–275 (1993)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002). doi:10.1007/s101070100263
de Oliveira, W.: Combining level and proximal bundle methods for convex optimization in energy problems. In: 3rd International Conference on Engineering Optimization—EngOpt, Rio de Janeiro, Mathematical Optimization Techniques, vol. 404, pp. 1–10 (2012). http://www.engopt.org/authors/404.html
de Oliveira, W., Sagastizábal, C.: Level bundle methods for oracles with on-demand accuracy. Optim. Methods Softw. 29(6), 1180–1209 (2014)
de Oliveira, W., Sagastizábal, C., Scheimberg, S.: Inexact bundle methods for two-stage stochastic programming. SIAM J. Optim. 21(2), 517–544 (2011). doi:10.1137/100808289
de Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Convex bundle methods in depth: a unified analysis for inexact oracles. Math. Program. 148(1–2), 241–277 (2014)
Fábián, C.: Bundle-type methods for inexact data. Cent. Eur. J. Oper. Res. 8, 35–55 (2000)
Frangioni, A.: Generalized bundle methods. SIAM J. Optim. 13(1), 117–156 (2002). doi:10.1137/S1052623498342186
Frangioni, A., Gorgone, E.: Bundle methods for sum-functions with ‘easy’ components: applications to multicommodity network design. Math. Program. 145(1–2), 133–161 (2014)
Goffin, J.L., Haurie, A., Vial, J.P.: Decomposition and nondifferentiable optimization with the projective algorithm. Manag. Sci. 38(2), 284–302 (1992). http://EconPapers.repec.org/RePEc:inm:ormnsc:v:38:y:1992:i:2:p:284-302
Goffin, J.L., Vial, J.P.: Interior points methods for nondifferentiable optimization. In: Proceedings of the Operations Research 1997 (Jena), pp. 35–49 (1998). http://EconPapers.repec.org/RePEc:fth:ehecge:97.24
Hintermüller, M.: A proximal bundle method based on approximate subgradients. Comput. Optim. Appl. 20, 245–266 (2001)
Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. No. 305–306 in Grund. der math. Wiss (two volumes). Springer, Berlin (1993)
Kiwiel, K.C.: Proximity control in bundle methods for convex nondiferentiable minimization. Math. Program. 46, 105–122 (1990)
Kiwiel, K.C.: Proximal level bubdle methods for convex nondiferentiable optimization, saddle-point problems and variational inequalities. Math. Program. 69(1), 89–109 (1995). doi:10.1007/BF01585554
Kiwiel, K.C.: A proximal bundle method with approximate subgradient linearizations. SIAM J. Optim. 16(4), 1007–1023 (2006)
Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69(1), 111–147 (1995). doi:10.1007/BF01585555
Lemaréchal, C., Sagastizábal, C.: Variable metric bundle methods: from conceptual to implementable forms. Math. Program. 76, 393–410 (1997)
Sagastizábal, C.: Composite proximal bundle method. Math. Program. 140(1), 189–233 (2013)
Sagastizábal, C., Solodov, M.: An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter. SIAM J. Optim. 16(1), 146–169 (2005). doi:10.1137/040603875
Solodov, M.: On approximations with finite precision in bundle methods for nonsmooth optimization. J. Optim. Theory Appl. 119(1), 151–165 (2003). doi:10.1023/B:JOTA.0000005046.70410.02
van Ackooij, W., de Oliveira, W.: Level bundle methods for constrained convex optimization with various oracles. Comput. Optim. Appl. 57(3), 555–597 (2014). doi:10.1007/s10589-013-9610-3
Acknowledgments
The authors would like to acknowledge helpful comments of Claudia Sagastizábal. The authors also thank the anonymous referees for constructive suggestions that considerably improved the original version of this article.
Author information
Authors and Affiliations
Corresponding author
Additional information
Mikhail Solodov is supported in part by CNPq Grant 302637/2011-7, by PRONEX-Optimization and by FAPERJ.
Rights and permissions
About this article
Cite this article
de Oliveira, W., Solodov, M. A doubly stabilized bundle method for nonsmooth convex optimization. Math. Program. 156, 125–159 (2016). https://doi.org/10.1007/s10107-015-0873-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-015-0873-6