Per-frame mAP Prediction for Continuous Performance Monitoring of Object Detection during Deployment
Date
2021
Authors
Rahman, Q.M.
Sunderhauf, N.
Dayoub, F.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Conference paper
Citation
Proceedings of the IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW 2021), 2021, pp.152-160
Statement of Responsibility
Quazi Marufur Rahman, Niko Sunderhauf, Feras Dayoub
Conference Name
IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW) (5 Jan 2021 - 9 Jan 2021 : Waikola, HI, USA)
Abstract
Performance monitoring of object detection is crucial for safety-critical applications such as autonomous vehicles that operate under varying and complex environmental conditions. Currently, object detectors are evaluated using summary metrics based on a single dataset that is assumed to be representative of all future deployment conditions. In practice, this assumption does not hold, and the performance fluctuates as a function of the deployment conditions. To address this issue, we propose an introspection approach to performance monitoring during deployment without the need for ground truth data. We do so by predicting when the per-frame mean average precision drops below a critical threshold using the detector’s internal features. We quantitatively evaluate and demonstrate our method’s ability to reduce risk by trading off making an incorrect decision by raising the alarm and absenting from detection.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
©2021 IEEE