Please use this identifier to cite or link to this item:
Scopus Web of Science® Altmetric
Type: Conference paper
Title: The VQA-machine: learning how to use existing vision algorithms to answer new questions
Author: Wang, P.
Wu, Q.
Shen, C.
van den Hengel, A.
Citation: Proceedings: 30th IEEE Conference on Computer Vision and Pattern Recognition, 2017 / vol.2017-January, pp.3909-3918
Publisher: IEEE
Issue Date: 2017
Series/Report no.: IEEE Conference on Computer Vision and Pattern Recognition
ISBN: 9781538604571
ISSN: 1063-6919
Conference Name: 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017) (21 Jul 2017 - 26 Jul 2017 : Honolulu)
Statement of
Peng Wang, Qi Wu, Chunhua Shen, Anton van den Hengel
Abstract: One of the most intriguing features of the Visual Question Answering (VQA) challenge is the unpredictability of the questions. Extracting the information required to answer them demands a variety of image operations from detection and counting, to segmentation and reconstruction. To train a method to perform even one of these operations accurately from {image, question, answer} tuples would be challenging, but to aim to achieve them all with a limited set of such training data seems ambitious at best. Our method thus learns how to exploit a set of external off-the-shelf algorithms to achieve its goal, an approach that has something in common with the Neural Turing Machine [10]. The core of our proposed method is a new co-attention model. In addition, the proposed approach generates human-readable reasons for its decision, and can still be trained end-to-end without ground truth reasons being given. We demonstrate the effectiveness on two publicly available datasets, Visual Genome and VQA, and show that it produces the state-of-the-art results in both cases.
Rights: © 2017 IEEE
RMID: 0030082775
DOI: 10.1109/CVPR.2017.416
Appears in Collections:Australian Institute for Machine Learning publications
Computer Science publications

Files in This Item:
There are no files associated with this item.

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.