Snap and Diagnose: An Advanced Multimodal Retrieval System for Identifying Plant Diseases in the Wild
Date
2024
Authors
Wei, T.
Chen, Z.
Yu, X.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Conference paper
Citation
Proceedings of the 6th ACM International Conference on Multimedia in Asia Mmasia, 2024, pp.131-1-131-3
Statement of Responsibility
Tianqi Wei, Zhi Chen, Xin Yu
Conference Name
MMAsia '24: ACM International Conference on Multimedia in Asia (MMAsia (3 Dec 2024 - 6 Dec 2024 : Auckland, New Zealand)
Abstract
Plant disease recognition is a critical task that ensures crop health and mitigates the damage caused by diseases. A handy tool that enables farmers to receive a diagnosis based on query pictures or text descriptions of suspicious plants is in high demand for initiating treatment before potential diseases spread further. In this paper, we develop a multimodal plant disease image retrieval system to support disease search based on either image or text prompts. Specifically, we utilize the largest in-the-wild plant disease dataset PlantWild, which includes over 18,000 images across 89 categories, to provide a comprehensive view of potential diseases relating to the query. Furthermore, cross-modal retrieval is achieved in the developed system, facilitated by a novel CLIP-based vision-language model that encodes both disease descriptions and disease images into the same latent space. Built on top of the retriever, our retrieval system allows users to upload either plant disease images or disease descriptions to retrieve the corresponding images with similar characteristics from the disease dataset to suggest candidate diseases for end users’ consideration.
School/Discipline
Dissertation Note
Provenance
Description
Poster presentation
Access Status
Rights
© 2024 Copyright held by the owner/author(s).