Please use this identifier to cite or link to this item:
Scopus Web of Science® Altmetric
Type: Conference paper
Title: Efficient computation of the Levenberg-Marquardt algorithm for feedforward networks with linear outputs
Author: De Chazal, P.
McDonnell, M.D.
Citation: Proceedings of International Joint Conference on Neural Networks, 2016, vol.2016-October, pp.68-75
Publisher: IEEE
Issue Date: 2016
Series/Report no.: IEEE International Joint Conference on Neural Networks (IJCNN)
ISBN: 9781509006199
ISSN: 2161-4393
Conference Name: International Joint Conference on Neural Networks (IJCNN) (24 Jul 2016 - 29 Jul 2016 : Vancouver, Canada)
Statement of
Philip de Chazal, Mark D. McDonnell
Abstract: An efficient algorithm for the calculation of the approximate Hessian matrix for the Levenberg-Marquardt (LM) optimization algorithm for training a single-hidden-layer feedforward network with linear outputs is presented. The algorithm avoids explicit calculation of the Jacobian matrix and computes the gradient vector and approximate Hessian matrix directly. It requires approximately 1/N the floating point operations of other published algorithms, where N is the number of network outputs. The required memory for the algorithm is also less than 1/N of the memory required for algorithms explicitly computing the Jacobian matrix. We applied our algorithm to two large-scale classification problems - the MNIST and the Forest Cover Type databases. Our results were within 0.5% of the best performance of systems using pixel values as inputs to a feedforward network for the MNIST database. Our results were achieved with a much smaller network than other published results. We achieved state-of-the-art performance for the Forest Cover Type database.
Keywords: Levenberg-Marquardt algorithm; feedforward neural networks; Gauss-Newton method; approximate Hessian calculation
Rights: © 2016 IEEE
DOI: 10.1109/IJCNN.2016.7727182
Grant ID:
Published version:
Appears in Collections:Aurora harvest 8
Electrical and Electronic Engineering publications

Files in This Item:
There are no files associated with this item.

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.