Skip to main content
Top

2016 | OriginalPaper | Chapter

Partial Linearization Based Optimization for Multi-class SVM

Authors : Pritish Mohapatra, Puneet Kumar Dokania, C. V. Jawahar, M. Pawan Kumar

Published in: Computer Vision – ECCV 2016

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We propose a novel partial linearization based approach for optimizing the multi-class svm learning problem. Our method is an intuitive generalization of the Frank-Wolfe and the exponentiated gradient algorithms. In particular, it allows us to combine several of their desirable qualities into one approach: (i) the use of an expectation oracle (which provides the marginals over each output class) in order to estimate an informative descent direction, similar to exponentiated gradient; (ii) analytical computation of the optimal step-size in the descent direction that guarantees an increase in the dual objective, similar to Frank-Wolfe; and (iii) a block coordinate formulation similar to the one proposed for Frank-Wolfe, which allows us to solve large-scale problems. Using the challenging computer vision problems of action classification, object recognition and gesture recognition, we demonstrate the efficacy of our approach on training multi-class svms with standard, publicly available, machine learning datasets.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
1.
go back to reference Collins, M., Globerson, A., Koo, T., Carreras, X., Bartlett, P.L.: Exponentiated gradient algorithms for conditional random fields and max-margin markov networks. J. Mach. Learn. Res. 9, 1775–1822 (2008)MathSciNetMATH Collins, M., Globerson, A., Koo, T., Carreras, X., Bartlett, P.L.: Exponentiated gradient algorithms for conditional random fields and max-margin markov networks. J. Mach. Learn. Res. 9, 1775–1822 (2008)MathSciNetMATH
2.
go back to reference Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. J. Mach. Learn. Res. 2, 265–292 (2001)MATH Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. J. Mach. Learn. Res. 2, 265–292 (2001)MATH
3.
go back to reference Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009) Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)
4.
go back to reference Engel, J.: Polytomous logistic regression. Statistica Neerlandica (1988) Engel, J.: Polytomous logistic regression. Statistica Neerlandica (1988)
6.
go back to reference Everingham, M., Van Gool, L., Williams, C., Winn, J., Zisserman, A.: The PASCAL visual object classes (VOC) challenge. IJCV 88, 303–338 (2010)CrossRef Everingham, M., Van Gool, L., Williams, C., Winn, J., Zisserman, A.: The PASCAL visual object classes (VOC) challenge. IJCV 88, 303–338 (2010)CrossRef
7.
go back to reference Fothergill, S., Mentis, H., Kohli, P., Nowozin, S.: Instructing people for training gestural interactive systems. In: SIGCHI Conference on Human Factors in Computing Systems (2012) Fothergill, S., Mentis, H., Kohli, P., Nowozin, S.: Instructing people for training gestural interactive systems. In: SIGCHI Conference on Human Factors in Computing Systems (2012)
8.
go back to reference Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval research logistics quarterly (1956) Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval research logistics quarterly (1956)
9.
go back to reference Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR (2014) Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR (2014)
10.
go back to reference Jaggi, M.: Revisiting frank-wolfe: projection-free sparse convex optimization. In: ICML (2013) Jaggi, M.: Revisiting frank-wolfe: projection-free sparse convex optimization. In: ICML (2013)
11.
go back to reference Joachims, T., Finley, T., Yu, C.J.: Cutting-plane training of structural SVMs. Mach. Learn. 77, 27–59 (2009). SpringerCrossRefMATH Joachims, T., Finley, T., Yu, C.J.: Cutting-plane training of structural SVMs. Mach. Learn. 77, 27–59 (2009). SpringerCrossRefMATH
12.
go back to reference Kivinen, J., Warmuth, M.: Relative loss bounds for multidimensional regression problems. JMLR 45, 301–329 (2001)MATH Kivinen, J., Warmuth, M.: Relative loss bounds for multidimensional regression problems. JMLR 45, 301–329 (2001)MATH
13.
go back to reference Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009) Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)
14.
go back to reference Lacoste-Julien, S., Jaggi, M., Schmidt, M., Pletscher, P.: Block-coordinate frank-wolfe for structural SVMs. In: ICML (2012) Lacoste-Julien, S., Jaggi, M., Schmidt, M., Pletscher, P.: Block-coordinate frank-wolfe for structural SVMs. In: ICML (2012)
15.
go back to reference Maji, S., Bourdev, L., Malik, J.: Action recognition from a distributed representation of pose and appearance. In: CVPR (2011) Maji, S., Bourdev, L., Malik, J.: Action recognition from a distributed representation of pose and appearance. In: CVPR (2011)
16.
go back to reference Malouf, R.: A comparison of algorithms for maximum entropy parameter estimation. In: Conference on Natural Language Learning (2002) Malouf, R.: A comparison of algorithms for maximum entropy parameter estimation. In: Conference on Natural Language Learning (2002)
17.
18.
go back to reference Quinlan, J.: Classification and regression trees. Programs for Machine Learning (2011) Quinlan, J.: Classification and regression trees. Programs for Machine Learning (2011)
19.
go back to reference Shalev-Shwartz, S., Singer, Y., Srebro, N., Cotter, A.: Pegasos: primal estimated sub-gradient solver for SVM. Math. Program. 127, 3–30 (2011). SpringerMathSciNetCrossRefMATH Shalev-Shwartz, S., Singer, Y., Srebro, N., Cotter, A.: Pegasos: primal estimated sub-gradient solver for SVM. Math. Program. 127, 3–30 (2011). SpringerMathSciNetCrossRefMATH
20.
go back to reference Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
21.
go back to reference Taskar, B., Guestrin, C., Koller, D.: Max-margin markov networks. In: NIPS (2004) Taskar, B., Guestrin, C., Koller, D.: Max-margin markov networks. In: NIPS (2004)
22.
go back to reference Wainwright, M.J., Jordan, M.: Graphical models, exponential families, and variational inference. Foundations and Trends\(\textregistered \) in Machine Learning (2008) Wainwright, M.J., Jordan, M.: Graphical models, exponential families, and variational inference. Foundations and Trends\(\textregistered \) in Machine Learning (2008)
23.
go back to reference Zhang, X., Saha, A., Vishwanathan, S.: Accelerated training of max-margin markov networks with kernels. Theoret. Comput. Sci. 519, 88–102 (2014). ElsevierMathSciNetCrossRefMATH Zhang, X., Saha, A., Vishwanathan, S.: Accelerated training of max-margin markov networks with kernels. Theoret. Comput. Sci. 519, 88–102 (2014). ElsevierMathSciNetCrossRefMATH
Metadata
Title
Partial Linearization Based Optimization for Multi-class SVM
Authors
Pritish Mohapatra
Puneet Kumar Dokania
C. V. Jawahar
M. Pawan Kumar
Copyright Year
2016
DOI
https://doi.org/10.1007/978-3-319-46454-1_51

Premium Partner