Please use this identifier to cite or link to this item:
Scopus Web of Science® Altmetric
Type: Conference paper
Title: What value do explicit high level concepts have in vision to language problems?
Author: Wu, Q.
Shen, C.
Liu, L.
Dick, A.
Van Den Hengel, A.
Citation: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016 / vol.2016-December, pp.203-212
Publisher: IEEE
Issue Date: 2016
Series/Report no.: IEEE Conference on Computer Vision and Pattern Recognition
ISBN: 9781467388511
ISSN: 1063-6919
Conference Name: 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (26 Jun 2016 - 01 Jul 2016 : Las Vegas, NV)
Statement of
Qi Wu, Chunhua Shen, Lingqiao Liu, Anthony Dick, Anton van den Hengel
Abstract: Much recent progress in Vision-to-Language (V2L) problems has been achieved through a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). This approach does not explicitly represent high-level semantic concepts, but rather seeks to progress directly from image features to text. In this paper we investigate whether this direct approach succeeds due to, or despite, the fact that it avoids the explicit representation of high-level information. We propose a method of incorporating high-level concepts into the successful CNN-RNN approach, and show that it achieves a significant improvement on the state-of-the-art in both image captioning and visual question answering. We also show that the same mechanism can be used to introduce external semantic information and that doing so further improves performance. We achieve the best reported results on both image captioning and VQA on several benchmark datasets, and provide an analysis of the value of explicit high-level concepts in V2L problems.
Rights: © 2016 IEEE
RMID: 0030055672
DOI: 10.1109/CVPR.2016.29
Appears in Collections:Computer Science publications

Files in This Item:
File Description SizeFormat 
RA_hdl_108049.pdfRestricted Access397.22 kBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.