Please use this identifier to cite or link to this item:
Scopus Web of Science® Altmetric
Type: Conference paper
Title: Parallel attention: a unified framework for visual object discovery through dialogs and queries
Author: Zhuang, B.
Wu, Q.
Shen, C.
Reid, I.
van den Hengel, A.
Citation: Proceedings: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), 2018 / pp.4252-4261
Publisher: IEEE
Issue Date: 2018
Series/Report no.: IEEE Conference on Computer Vision and Pattern Recognition
ISBN: 9781538664209
ISSN: 2575-7075
Conference Name: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (18 Jun 2018 - 23 Jun 2018 : Salt Lake City, UT)
Statement of
Bohan Zhuang, Qi Wu, Chunhua Shen, Ian Reid, Anton van den Hengel
Abstract: Recognising objects according to a pre-defined fixed set of class labels has been well studied in the Computer Vision. There are a great many practical applications where the subjects that may be of interest are not known beforehand, or so easily delineated, however. In many of these cases natural language dialog is a natural way to specify the subject of interest, and the task achieving this capability (a.k.a, Referring Expression Comprehension) has recently attracted attention. To this end we propose a unified framework, the ParalleL AttentioN (PLAN) network, to discover the object in an image that is being referred to in variable length natural expression descriptions, from short phrases query to long multi-round dialogs. The PLAN network has two attention mechanisms that relate parts of the expressions to both the global visual content and also directly to object candidates. Furthermore, the attention mechanisms are recurrent, making the referring process visualizable and explainable. The attended information from these dual sources are combined to reason about the referred object. These two attention mechanisms can be trained in parallel and we find the combined system outperforms the state-of-art on several benchmarked datasets with different length language input, such as RefCOCO, RefCOCO+ and GuessWhat?!.
Rights: © 2018 IEEE
RMID: 0030108883
DOI: 10.1109/CVPR.2018.00447
Grant ID:
Appears in Collections:Australian Institute for Machine Learning publications
Computer Science publications

Files in This Item:
File Description SizeFormat 
hdl_120064.pdfSubmitted version2.11 MBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.