Liao, Z.Wu, Q.Shen, C.Van Den Hengel, A.Verjans, J.Cappellato, L.Eickhoff, C.Ferro, N.Névéol, A.2021-09-232021-09-232020CEUR Workshop Proceedings, 2020 / Cappellato, L., Eickhoff, C., Ferro, N., Névéol, A. (ed./s), vol.2696, pp.1-141613-0073https://hdl.handle.net/2440/132209Session - ImageCLEF: Multimedia Retrieval in Medicine, Lifelogging, and Internet.In this paper, we describe our contribution to the 2020 ImageCLEF Medical Domain Visual Question Answering (VQA-Med) challenge. Our submissions scored first place on the VQA challenge leaderboard, and also the first place on the associated Visual Question Generation (VQG) challenge leaderboard. Our VQA approach was developed using a knowledge inference methodology called Skeleton-based Sentence Mapping (SSM). Using all the questions and answers, we derived a set of classifiable tasks and inferred the corresponding labels. As a result, we were able to transform the VQA task into a multi-task image classification problem which allowed us to focus on the image modelling aspect. We further propose a class-wise and task-wise normalization facilitating optimization of multiple tasks in a single network. This enabled us to apply a multi-scale and multi-architecture ensemble strategy for robust prediction. Lastly, we positioned the VQG task as a transfer learning problem using the VGA task trained models. The VQG task was also solved using classification.enCopyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).Visual Question Answering; Visual Question Generation; Knowledge Inference; Deep Neural Networks; Skeleton-based Sentence Mapping; Class-wise and Task-wise NormalizationConference paper2021-09-22554527Liao, Z. [0000-0001-9965-4511]Wu, Q. [0000-0003-3631-256X]Van Den Hengel, A. [0000-0003-3027-8364]Verjans, J. [0000-0002-8336-6774]