A Wearable Assistive Technology for the Visually Impaired with Door Knob Detection and Real-Time Feedback for Hand-to-Handle Manipulation

Liang Niu, Cheng Qian, John Ross Rizzo, Todd Hudson, Zichen Li, Shane Enright, Eliot Sperling, Kyle Conti, Edward Wong, Yi Fang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The visually impaired are consistently faced with mobility restrictions due to the lack of truly accessible environments. Even in structured settings, people with low vision may still have trouble navigating efficiently and safely due to hallway and threshold ambiguity. Assistive technologies that are currently available do not provide door and door-handle object detections nor do they concretely help the visually impaired reaching towards the object. In this paper, we propose an AI-driven wearable assistive technology that integrates door handle detection, user's real-time hand position in relation to this targeted object, and audio feedback for 'joy stick-like command' for acquisition of the target and subsequent hand-to-handle manipulation. When fully envisioned, this platform will help end users locate doors and door handles and reach them with feedback, enabling them to travel safely and efficiently when navigating through environments with thresholds. Compared to the usual computer vision models, the one proposed in this paper requires significantly fewer computational resources, which allows it to pair with a stereoscopic camera running on a small graphics processing unit (GPU). This permits us to take advantage of its convenient portability. We also introduce a dataset containing different types of door handles and door knobs with bounding-box annotations, which can be used for training and testing in future research.

Original languageEnglish (US)
Title of host publicationProceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1500-1508
Number of pages9
Volume2018-January
ISBN (Electronic)9781538610343
DOIs
Publication statusPublished - Jan 19 2018
Event16th IEEE International Conference on Computer Vision Workshops, ICCVW 2017 - Venice, Italy
Duration: Oct 22 2017Oct 29 2017

Other

Other16th IEEE International Conference on Computer Vision Workshops, ICCVW 2017
CountryItaly
CityVenice
Period10/22/1710/29/17

    Fingerprint

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

Niu, L., Qian, C., Rizzo, J. R., Hudson, T., Li, Z., Enright, S., ... Fang, Y. (2018). A Wearable Assistive Technology for the Visually Impaired with Door Knob Detection and Real-Time Feedback for Hand-to-Handle Manipulation. In Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017 (Vol. 2018-January, pp. 1500-1508). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCVW.2017.177