Please use this identifier to cite or link to this item: https://hdl.handle.net/2440/61402
Type: Journal article
Title: A quasi-newton approach to nonsmooth convex optimization problems in machine learning
Author: Yu, J.
Vishwanathan, S.
Gunter, S.
Schraudolph, N.
Citation: Journal of Machine Learning Research, 2010; 11:1145-1200
Publisher: MIT Press
Issue Date: 2010
ISSN: 1532-4435
1533-7928
Statement of
Responsibility: 
Jin Yu, S.V.N. Vishwanathan, Simon Günter and Nicol N. Schraudolph
Abstract: We extend the well-known BFGS quasi-Newton method and its memory-limited variant LBFGS to the optimization of nonsmooth convex objectives. This is done in a rigorous fashion by generalizing three components of BFGS to subdifferentials: the local quadratic model, the identification of a descent direction, and the Wolfe line search conditions. We prove that under some technical conditions, the resulting subBFGS algorithm is globally convergent in objective function value. We apply its memory-limited variant (subLBFGS) to L2-regularized risk minimization with the binary hinge loss. To extend our algorithm to the multiclass and multilabel settings, we develop a new, efficient, exact line search algorithm. We prove its worst-case time complexity bounds, and show that our line search can also be used to extend a recently developed bundle method to the multiclass and multilabel settings. We also apply the direction-finding component of our algorithm to L1-regularized risk minimization with logistic loss. In all these contexts our methods perform comparable to or better than specialized state-of-the-art solvers on a number of publicly available data sets. An open source implementation of our algorithms is freely available.
Keywords: BFGS
variable metric methods
Wolfe conditions
subgradient
risk minimization
hinge loss
multiclass, multilabel
bundle methods
BMRM
OCAS
OWL-QN
Rights: (c) 2010 Jin Yu, S.V.N. Vishwanathan, Simon Günter and Nicol N. Schraudolph.
Appears in Collections:Aurora harvest 5
Computer Science publications

Files in This Item:
There are no files associated with this item.


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.