Domain Generalization and its Application in Deepfake Detection
Date
2024
Authors
Chen, Liang
Editors
Advisors
Liu, Lingqiao
Gong, Dong
Gong, Dong
Journal Title
Journal ISSN
Volume Title
Type:
Thesis
Citation
Statement of Responsibility
Conference Name
Abstract
Despite the continuous emergence of new machine learning systems, how to effectively deploy deep models in the real-world environment remains challenging. This is mainly due to the limited generability of current deep models: most current models will suffer severe performance degradation when there is a distribution shift between the training and test environments. This thesis aims to ease the generalization problem from both the general machine learning and specific downstream application perspectives. For general machine learning systems, we propose two effective domain generalization (DG) methods given different settings where training samples are with and without domain information. Specifically, in the case when domain information is available, we suggest learning experts that specialized in different domains besides only learning the target model, so that the experts can provide professional guidance to refine the output of the target model in the training phase; In another case where domain information is unavailable, we suggest improving the vanilla test-time training (TTT) strategy for DG. Specifically, we propose a learnable consistency loss, that can align with the main loss via learnable parameters, for the TTT task, and we also suggest using additional adaptive parameters, which are the only updated parameters during the test phase, for the trained mode. By conducting experiments on different benchmarks, we show that the two ideas can both improve generalization for the baseline model. For the specific downstream application, we design two robust deep algorithms to tackle the deepfake detection task. Our first design is based on a simple principle: a generalizable representation should be sensitive to diverse types of forgeries. Two strategies are conducted to improve generalizability based on the principle: the “diversity” of forgeries is enriched by synthesizing augmented forgeries with a pool of forgery configurations; the “sensitivity” to the forgeries is strengthened by enforcing the model to predict the forgery configurations; Our second effort involves a new learning paradigm specially designed for the task, which is derived from the TTT idea. The key idea is to construct a test-sample-specific auxiliary task to update the model before applying it to the sample, and we fulfill the task by synthesizing pseudo-training samples from each test image and creating a test-time training objective to update the model. By conducting experiments on various deepfake detection datasets, we show that the proposed two methods perform favorably against existing arts.
School/Discipline
School of Computer Science
Dissertation Note
Thesis (Ph.D.) -- University of Adelaide, School of Computer and Mathematical Sciences, 2024
Provenance
This electronic version is made publicly available by the University of Adelaide in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. This thesis may incorporate third party material which has been used by the author pursuant to Fair Dealing exceptions. If you are the owner of any included third party copyright material you wish to be removed from this electronic version, please complete the take down form located at: http://www.adelaide.edu.au/legals