Overcoming language priors in VQA via decomposed linguistic representations

Simple item page
dc.contributor.authorJing, C.
dc.contributor.authorWu, Y.
dc.contributor.authorZhang, X.
dc.contributor.authorJia, Y.
dc.contributor.authorWu, Q.
dc.contributor.conferenceAAAI Conference on Artificial Intelligence (AAAI) (7 Feb 2020 - 12 Feb 2020 : New York, USA)
dc.date.issued2020
dc.description.abstractMost existing Visual Question Answering (VQA) models overly rely on language priors between questions and answers. In this paper, we present a novel method of language attention-based VQA that learns decomposed linguistic representations of questions and utilizes the representations to infer answers for overcoming language priors. We introduce a modular language attention mechanism to parse a question into three phrase representations: type representation, object representation, and concept representation. We use the type representation to identify the question type and the possible answer set (yes/no or specific concepts such as colors or numbers), and the object representation to focus on the relevant region of an image. The concept representation is verified with the attended region to infer the final answer. The proposed method decouples the language-based concept discovery and vision-based concept verification in the process of answer inference to prevent language priors from dominating the answering process. Experiments on the VQA-CP dataset demonstrate the effectiveness of our method.
dc.description.statementofresponsibilityChenchen Jing, Yuwei Wu, Xiaoxun Zhang, Yunde Jia, Qi Wu
dc.identifier.citationProceedings of the ... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence, 2020, vol.34, iss.7, pp.11181-11188
dc.identifier.doi10.1609/aaai.v34i07.6776
dc.identifier.isbn9781577358350
dc.identifier.issn2159-5399
dc.identifier.issn2374-3468
dc.identifier.orcidWu, Q. [0000-0003-3631-256X]
dc.identifier.urihttps://hdl.handle.net/2440/133677
dc.language.isoen
dc.publisherAAAI
dc.publisher.placeonline
dc.relation.granthttp://purl.org/au-research/grants/arc/DE190100539
dc.relation.ispartofseriesAAAI Conference on Artificial Intelligence
dc.rights© 2020, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
dc.source.urihttps://aaai.org/Library/AAAI/aaai20contents-issue07.php
dc.titleOvercoming language priors in VQA via decomposed linguistic representations
dc.typeConference paper
pubs.publication-statusPublished

Files

Collections

Computer Science publications