Boosting Manifold Ranking for image retrieval by mining query log repeatedly

Abstract

Manifold Ranking (MR) is one popular and successful technique for relevance feedback in content-based image retrieval (CBIR). However, existing MR methods have two main drawbacks. First, the affinity matrix used by MR is computed purely based on the visual features of images, which fails to accurately capture the semantic structure of image database. Second, the existing MR methods often suffer from the “cold start” problem where the feedback example set is quite small. In this paper, we propose a novel scheme that double exploits the query log in MR to address the drawbacks. In details, the correlation between each pair of database images is first estimated based on a query log, which serves to adjust the affinity matrix towards semantic structure. Then, the relevance score of each database image to the user’s query is further inferred from the query log, which could be used to produce more pseudo-labeled examples to handle the “cold start” problem. An empirical study shows that the proposed scheme is more effective than the state-of-the-art approaches.