Goal-oriented visual question generation via intermediate rewards

Simple item page
dc.contributor.authorZhang, J.
dc.contributor.authorWu, Q.
dc.contributor.authorShen, C.
dc.contributor.authorZhang, J.
dc.contributor.authorLu, J.
dc.contributor.authorvan den Hengel, A.
dc.contributor.conference15th European Conference on Computer Vision (ECCV 2018) (8 Sep 2018 - 14 Sep 2018 : Munich)
dc.contributor.editorFerrari, V.
dc.contributor.editorHebert, M.
dc.contributor.editorSminchisescu, C.
dc.contributor.editorWeiss, Y.
dc.date.issued2018
dc.description.abstractDespite significant progress in a variety of vision-and-language problems, developing a method capable of asking intelligent, goal-oriented questions about images is proven to be an inscrutable challenge. Towards this end, we propose a Deep Reinforcement Learning framework based on three new intermediate rewards, namely goal-achieved, progressive and informativeness that encourage the generation of succinct questions, which in turn uncover valuable information towards the overall goal. By directly optimizing for questions that work quickly towards fulfilling the overall goal, we avoid the tendency of existing methods to generate long series of inane queries that add little value. We evaluate our model on the GuessWhat?! dataset and show that the resulting questions can help a standard ‘Guesser’ identify a specific object in an image at a much higher success rate.
dc.description.statementofresponsibilityJunjie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, Jianfeng Lu and Anton van den Hengel
dc.identifier.citationLecture Notes in Artificial Intelligence, 2018 / Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (ed./s), vol.Lecture Notes in Computer Science; vol. 11209, pp.189-204
dc.identifier.doi10.1007/978-3-030-01228-1_12
dc.identifier.isbn9783030012274
dc.identifier.issn0302-9743
dc.identifier.issn1611-3349
dc.identifier.orcidWu, Q. [0000-0003-3631-256X]
dc.identifier.orcidvan den Hengel, A. [0000-0003-3027-8364]
dc.identifier.urihttp://hdl.handle.net/2440/116151
dc.language.isoen
dc.publisherSpringer
dc.relation.ispartofseriesLecture Notes in Computer Science
dc.rights© Springer Nature Switzerland AG 2018
dc.source.urihttps://doi.org/10.1007/978-3-030-01228-1_12
dc.subjectGoal-oriented
dc.subjectVQG
dc.subjectintermediate rewards
dc.titleGoal-oriented visual question generation via intermediate rewards
dc.typeConference paper
pubs.publication-statusPublished

Files

Collections

Computer Science publications
Australian Institute for Machine Learning publications