EchoGuide

Visually impaired people navigate the world well with a cane or a guide dog, but neither tells you what an object is or exactly where it is on a table or shelf. Finding a specific misplaced item in a familiar room can take minutes of searching by hand.

EchoGuide came out of the same ETH Zurich Build2gether 2.0 Inclusive Innovation Challenge as RescueTap, and out of the same starting question: what does independence actually require, beyond avoiding obstacles? Existing assistive devices are almost all about mobility — walking safely. Almost none of them answer 'where is my coffee cup.'

Sole designer and engineer — model selection and deployment, the audio interface, and the physical design of the wearable.

A Raspberry Pi 4 wearable cap with a Logitech HD webcam sees the room in real time; a quantized TensorFlow Lite SSD MobileNet v1 model (trained on COCO classes) identifies objects in the frame; the result is spoken back through headphones. Direction is described using a clock-face convention — 'cup at two o'clock' — and distance is conveyed by beeps that speed up as the object gets closer. Two verbosity modes let the user choose between just object names or object-plus-colour, the latter using a KDTree nearest-match against the CSS3 colour table.

• The industrial design decision I'm proudest of on the whole site: the camera is the only visible hardware, and every wire is sewn into the fabric of the cap. A device that visibly marks its wearer as impaired is a device people stop wearing in public — so the cap had to look like a cap first
• Chose a clock-face + beep-frequency encoding over spoken coordinates because it's the same mental model canes and orientation training already use — the device should extend a skill the user already has, not teach a new one
• Built two verbosity levels because 'basic' is faster to parse when moving, and 'detailed' (with colour) is more useful when stationary and deciding between similar objects — the same feedback density isn't right for every moment

TensorFlow Lite, SSD MobileNet v1 (quantized)

small and fast enough to run object detection on a Pi 4 in real time — a full-size model would be too slow to be useful while walking

OpenCV

video capture and frame handling, threaded separately from inference so audio feedback doesn't stall waiting on a frame

pyttsx3 + SpeechRecognition

on-device text-to-speech and voice input — no network round-trip, which matters when the device needs to work as reliably indoors as it does outside coverage

webcolors + scipy KDTree

nearest-neighbour colour naming against the CSS3 table, fast enough to run per-detection without noticeable lag

• Video capture runs on its own thread so a slow inference pass never blocks the camera loop — a common failure mode in single-threaded CV demos that would have made the audio feedback feel laggy and unreliable
• Object detection outputs are COCO class labels; for each detection I additionally sample the bounding-box region for a dominant colour and match it via KDTree against the CSS3 name table, so 'detailed' mode can say 'red cup' rather than just 'cup'

• SSD MobileNet is fast but not always precise on partially occluded objects — in a cluttered room it sometimes named the wrong item in a cluster. A larger model would fix this at the cost of the real-time performance the whole device depends on; I chose speed, and it's the trade-off I'd revisit first with more compute budget
• Getting the audio feedback timing right — frequent enough to be useful, sparse enough not to overwhelm — took more iteration than the computer vision did. This is a UX problem wearing a technical costume

• Awarded a Lucky Draw prize for two submissions at the ETH Zurich Build2gether 2.0 Inclusive Innovation Challenge
• Recognised in the top 1% of 664 global participants

The hardest part of this project was never the model — it was deciding how much information a person can absorb through one sense while their attention is on walking safely. That's a product question, not a machine-learning question.