Improving the Performance and Robustness of NLP models with Contrastive Data and Learning
Date
2025
Authors
Zhuang, Haojie
Editors
Advisors
Zhang, Wei
Sheng, Michael (Macquarie University)
Sheng, Michael (Macquarie University)
Journal Title
Journal ISSN
Volume Title
Type:
Thesis
Citation
Statement of Responsibility
Conference Name
Abstract
Contrastive learning enables representation learning by optimizing the models to distinguish between similar and dissimilar examples, which leads to more high-quality, robust and generalizable representations. Contrastive learning has also been widely used to improve the performance and robustness of natural language processing (NLP) models across different tasks. While extensively explored, existing contrastive learning methods still face challenges and issues, such as how to utilize the contrastive data more efficiently for language generation or representation, address false negatives issue and ensure robust evaluation with contrastive data. This dissertation investigates and extends the application of contrastive learning to diverse NLP tasks to improve the performance and robustness, including language generation, representation learning, faithfully explainable recommendation and robust evaluation. Language Generation: We focus on summarization, which is one of the most important language generation tasks. We introduce a novel method to utilize contrastive data and learning for unsupervised summarization and propose two models to improve the quality of the generated summaries. Representation Learning: We study the language and multimodal representation learning with contrastive learning, and present two effective contrastive data processing methods for contrastive loss to improve the quality and robustness of the representations in language and multimodal learning. Faithfully Explainable Recommendation: We are the first to introduce contrastive data and learning to enhance the faithfulness and robustness of explanation in recommender system, and propose a simple but effective method to alleviate the hallucination issue in explainable recommendation. Robust Evaluation: We provide a comprehensive research framework for multimodal summarization evaluation, including (1) automatic evaluation metric; (2) metaevaluation benchmark (with contrastive and diverse data for robustness) and analysis; (3) cognitive biases analysis in human evaluation (for robust and unbiased evaluation). We believe our work would be a valuable resource for the multimodal summarization and robust evaluation research community. Through extensive theoretical studies and empirical analysis, this dissertation broadens the applicability of contrastive data and learning in various NLP tasks, and demonstrates that leveraging contrastive data and learning is a powerful method for NLP. Lastly and importantly, we highlight the impact of our contributions in enhancing NLP research and development, paving the way for more efficient and robust AI systems.
School/Discipline
School of Computer and Mathematical Sciences
Dissertation Note
Thesis (Ph.D.) -- University of Adelaide, School of Computer and Mathematical Sciences, 2025
Provenance
This electronic version is made publicly available by the University of Adelaide in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. This thesis may incorporate third party material which has been used by the author pursuant to Fair Dealing exceptions. If you are the owner of any included third party copyright material you wish to be removed from this electronic version, please complete the take down form located at: http://www.adelaide.edu.au/legals