Weakly Supervised Localization for Censor Aware Survival Prediction from Medical Images

Abstract

In recent years deep learning systems have shown promising results in healthcare, producing outstanding accuracy in several medical image analysis tasks, such as disease classification and survival prediction. However, there are currently limited examples of such techniques being successfully deployed into clinical practice given the high level of accountability needed for these systems. In fact, for a successful deployment, these deep learning systems need to provide explanations for their outputs in order to gain the trust of clinicians. In the domain of medical image analysis, one form of explanation is based on the localization of regions of interest that are associated with the model’s predictions. This type of explanation can be achieved with the fully supervised training of visual object detectors that require annotated data sets, where the visual objects of interest are localized in the training images. Unfortunately, these data sets are expensive to acquire and prone to contain label noise. Given this limitation, the community started to investigate an alternative approach based on weakly supervised object localization (WSOL), where the model is able to associate input image regions with particular predictions without requiring fully annotated data sets. However, while there has been strong interest for the development of WSOL methods for disease classification problems, WSOL has received relatively little attention for survival prediction problems, but we argue in this thesis that the field should study WSOL for survival prediction more in depth because of the potential benefits it offers for treatment decision and patient care. Motivated by the limitations mentioned above, we focus this thesis on the study of model interpretability for survival prediction problems. First, we study WSOL methods on a multi-label classification problem using chest x-rays (CXR), where we introduce a novel method for weakly supervised localization combining region proposals and saliency map approaches. Then, we propose an interpretability method for survival prediction using a data set of brain tumor from magnetic resonance imaging (MRI) that does not contain censored data. For this problem, we propose a new post-hoc approach to output survival time predictions together with saliency maps to localize tumors in the image. Next, we study the challenging problem of survival prediction with censored and uncensored data from CXR of asymptomatic patients exposed to known risk factors of lung cancer (e.g., cigarette smoking). For this problem, we introduce a new semi-supervised training method, based on pseudo-labeling, to effectively train with all censored and uncensored data. Finally, building on top of our previous works, we propose a new WSOL survival prediction method trained from censored and uncensored CXR images of asymptomatic patients to automatically localize future cancerous lesions. Experiments show that this method is able to locate regions where lung cancer tumors will develop years before to the first cancer diagnosis.