Towards Better Efficiency and Generalization in Imitation Learning: A Causal Perspective
Date
2024
Authors
Jabri, Mohamed Khalil
Editors
Advisors
Shi, Javen Qinfeng
Abbasnejad, Ehsan
Abbasnejad, Ehsan
Journal Title
Journal ISSN
Volume Title
Type:
Thesis
Citation
Statement of Responsibility
Conference Name
Abstract
Imitation Learning, also known as Learning from Demonstrations, has emerged as a practical alternative to reinforcement learning, mitigating the intricate challenges associated with reward engineering in the latter. However, imitation learning agents often face limitations that hinder their effectiveness in realistic scenarios. Sample efficiency and generalization pose notable challenges among these limitations. Concurrently, there has been an increasing acknowledgment of causality’s significance in improving learning-based approaches, resulting in its recent prominence within the machine learning community. This dissertation explores the potential of causality-inspired approaches to address the aforementioned challenges in imitation learning through two distinct contributions. The first contribution introduces a novel method to enhance goal-conditioned imitation learning using Structural Causal Models (SCMs) and counterfactual data. We leverage SCMs as a formalism to understand the inherent causal relationships between different variables governing expert behavior. This enables the generation of counterfactual data, which we utilize to learn improved reward functions with reduced data, thereby enhancing the agent’s efficiency. The second contribution focuses on identifying causal features that remain consistent across different environments. Unlike many works on domain generalization, our method is equally applicable to Reinforcement Learning and Imitation Learning, eschewing the need for domain supervision and remaining agnostic to data modality, rendering it broadly applicable. Through empirical evaluation, this dissertation establishes the efficacy of causalityinspired approaches in advancing imitation learning capabilities. The proposed methodologies not only contribute to overcoming some fundamental limitations of existing imitation learning algorithms but also provide valuable insights into the broader application of causality in machine learning.
School/Discipline
School of Computer and Mathematical Sciences
Dissertation Note
Thesis (Ph.D.) -- University of Adelaide, School of Computer and Mathematical Sciences, 2024
Provenance
This electronic version is made publicly available by the University of Adelaide in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. This thesis may incorporate third party material which has been used by the author pursuant to Fair Dealing exceptions. If you are the owner of any included third party copyright material you wish to be removed from this electronic version, please complete the take down form located at: http://www.adelaide.edu.au/legals