Towards Pedestrian Safety Augmented Reality System
Date
2024
Authors
Wu, Renjie
Editors
Advisors
Chen, Tim
Dayoub, Feras
Dayoub, Feras
Journal Title
Journal ISSN
Volume Title
Type:
Thesis
Citation
Statement of Responsibility
Conference Name
Abstract
Pedestrian safety has always been a critical focus in the field of Human-Computer Interaction (HCI), gaining increasing significance with the growing popularity of consumergrade Augmented Reality (AR) headsets. While AR headsets provide users with a more immersive experience, they can also present significant distractions, heightening the risks for pedestrians who may overlook oncoming vehicles. To build an AR system for pedestrian safety, it is essential that the system effectively preserves and warns distracted users about approaching vehicles. Therefore, to achieve this goal, this thesis addresses two research questions: (1) how to effectively warn distracted users of approaching vehicles, and (2) how to utilize sensors on consumer-grade AR headsets to assist users in locating surrounding sound-making vehicles. Concerning the first question, previous researchers have explored various real-time AR warnings but often neglect the context of distraction. We conducted a Virtual Reality experiment to investigate the impact of the matching question between visual and auditory modality of distraction and warning on pedestrian street-crossing behavior. Our findings suggest that mismatched modality warning (e.g., visual distraction with auditory warning or auditory distraction with visual warning) should be applied to pedestrians wearing AR headsets, as it leads to improved user behavior, such as significantly faster reaction times, reduced walking speed, and increased scanning range after receiving the warning. Subsequent interviews also showed that users preferred mismatched modality warnings. Regarding the second question, previous work has employed omnidirectional cameras or additional rear cameras to locate vehicles outside the user’s field of view (FoV). However, due to the limited camera FoV on existing consumer-grade AR devices, locating out-of-view vehicles is exceedingly challenging. Taking advantage of rapidly evolving deep learning techniques, we propose the out-of-view objects semantic segmentation task, introducing the Segment beyond View (SBV) framework to solve this question. SBV utilizes the front-facing camera with limited FoV and ambient microphones to enable semantic segmentation of sound-making vehicles in the panorama. We envision that future AR system should incorporate the mismatched modality warnings and SBV framework to mitigate safety risks for pedestrians.
School/Discipline
School of Computer and Mathematical Sciences
Dissertation Note
Thesis (MPhil) -- University of Adelaide, School of Computer and Mathematical Sciences, 2024
Provenance
This electronic version is made publicly available by the University of Adelaide in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. This thesis may incorporate third party material which has been used by the author pursuant to Fair Dealing exceptions. If you are the owner of any included third party copyright material you wish to be removed from this electronic version, please complete the take down form located at: http://www.adelaide.edu.au/legals