Please use this identifier to cite or link to this item:
|Scopus||Web of Science®||Altmetric|
|Title:||Visual question answering as reading comprehension|
Van Den Hengel, A.
|Citation:||Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019 / vol.2019-June, pp.6312-6321|
|Series/Report no.:||IEEE Conference on Computer Vision and Pattern Recognition|
|Conference Name:||IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (15 Jun 2019 - 20 Jun 2019 : Long Beach, USA)|
|Hui Li, Peng Wang, Chunhua Shen, Anton van den Hengel|
|Abstract:||Visual question answering (VQA) demands simultaneous comprehension of both the image visual content and natural language questions. In some cases, the reasoning needs the help of common sense or general knowledge which usually appear in the form of text. Current methods jointly embed both the visual information and the textual feature into the same space. Nevertheless, how to model the complex interactions between the two different modalities is not an easy work. In contrast to struggling on multimodal feature fusion, in this paper, we propose to unify all the input information by natural language so as to convert VQA into a machine reading comprehension problem. With this transformation, our method not only can tackle VQA datasets that focus on observation based questions, but can also be naturally extended to handle knowledge-based VQA which requires to explore large-scale external knowledge base. It is a step towards being able to exploit large volumes of text and natural language processing techniques to address VQA problem. Two types of models are proposed to deal with open-ended VQA and multiple-choice VQA respectively. We evaluate our models on three VQA benchmarks. The comparable performance with the state-of-the-art demonstrates the effectiveness of the proposed method.|
|Appears in Collections:||Australian Institute for Machine Learning publications|
Files in This Item:
|hdl_127237.pdf||Submitted version||5.37 MB||Adobe PDF||View/Open|
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.