Walk Assist: An Integrated Arcore and Vision-Language Based Mobile Walking Assistance System For Blind And Low-Vision Pedestrians

This paper presents WalkAssist, an Android-based walking assistance system designed for blind and low-vision pedestrians. The system integrates ARCore spatial sensing, YOLO-based object segmentation, one-shot OCR, Gemini 2.5 Flash-Lite API scene description, map-based pedestrian route guidance, and multimodal feedback into a single mobile prototype. ARCore camera pose, hit tests, raw depth, and depth confidence are utilized to estimate the walking zone and nearby obstacle distance. YOLO26n-seg detects objects and provides segmentation masks, which are combined with depth samples to improve object-level distance awareness. OCR reads visible text when requested, while the Gemini 2.5 Flash-Lite API provides natural-language scene descriptions for additional context. The feedback layer delivers information through an on-screen overlay, text-to-speech, vibration, tones, and accessibility announcements. The implemented prototype demonstrates the feasibility of combining multiple perception and feedback modules on a smartphone to support pedestrian awareness. Development also revealed practical challenges, including ARCore depth instability, low-light sensitivity, and VLM latency, which were mitigated through confidence filtering, fallback depth sampling, output prioritization, and local walking-zone feedback prior to remote VLM responses.