Mitigating the Impact of Inaccurate Feedback in Dynamic Learning-to-Rank: A Study of Overlooked Interesting Items
Date
2024
Authors
Zhang, C.
Chen, W.
Zhang, W.
Xu, M.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Journal article
Citation
ACM Transactions on Intelligent Systems and Technology, 2024; 16(1):5-1-5-26
Statement of Responsibility
Chenhao Zhang, Weitong Chen, Wei Zhang, Miao Xu
Conference Name
DOI
Abstract
Dynamic Learning-to-Rank (DLTR) is a method of updating a ranking policy in real time based on user feedback, which may not always be accurate. Although previous DLTR work has achieved fair and unbiased DLTR under inaccurate feedback, they face the tradeoff between fairness and user utility and also have limitations in the setting of feeding items. Existing DLTR works improve ranking utility by eliminating bias from inaccurate feedback on observed items, but the impact of another pervasive form of inaccurate feedback, overlooked or ignored interesting items, remains unclear. For example, users may browse the rankings too quickly to catch interesting items or miss interesting items because the snippets are not optimized enough. This phenomenon raises two questions: (i) Will overlooked interesting items affect the ranking results? and (ii) Is it possible to improve utility without sacrificing fairness if these effects are eliminated? These questions are particularly relevant for small and medium-sized retailers who are just starting out and may have limited data, leading to the use of inaccurate feedback to update their models. In this article, we find that inaccurate feedback in the form of overlooked interesting items has a negative impact on DLTR performance in terms of utility. To address this, we treat the overlooked interesting items as noise and propose a novel DLTR method, the Co-teaching Rank (CoTeR), that has good utility and fairness performance when inaccurate feedback is present in the form of overlooked interesting items. Our solution incorporates a co-teaching-based component with a customized loss function and data sampling strategy, as well as a mean pooling strategy to further accommodate newly added products without historical data. Through experiments, we demonstrate that CoTeR not only enhances utilities but also preserves ranking fairness and can smoothly handle newly introduced items.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
©2024 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution International 4.0 License.