Skip to content
All work
AI & Computer VisionBuild2gether — ETH Zurich

EchoGuide

A wearable cap that tells visually impaired users what is around them, and where.

Role
Sole designer and engineer — model, audio interface, wearable design
Timeline
Mar 2024 — Sep 2024
Stack
  • Python
  • TensorFlow Lite
  • OpenCV
  • SSD MobileNet
  • Raspberry Pi 4
  • pyttsx3
  • SpeechRecognition
Links
Full build guide (opens in a new tab)

In short

Problem
A cane or guide dog helps someone move safely, but neither says what an object is or where it sits.
My contribution
Sole designer and engineer: on-device object detection, the audio interface and the wearable itself.
Key decision
The camera is the only visible hardware — a device that marks its wearer as impaired is one people stop wearing.
Outcome
A wearable that names objects with clock-face direction and beep-coded distance; top 1% of 664 at ETH Zurich Build2gether 2.0.

The problem

What was broken, for whom

Visually impaired people navigate the world well with a cane or a guide dog, but neither tells you what an object is or exactly where it is on a table or shelf. Finding a specific misplaced item in a familiar room can take minutes of searching by hand.

Context & constraints

EchoGuide came out of the same ETH Zurich Build2gether 2.0 Inclusive Innovation Challenge as RescueTap, and out of the same starting question: what does independence actually require, beyond avoiding obstacles? Existing assistive devices are almost all about mobility — walking safely. Almost none of them answer 'where is my coffee cup.'

My role

What I actually did

Sole designer and engineer — model selection and deployment, the audio interface, and the physical design of the wearable.

Key decisions

Trade-offs made on purpose

  1. Decision 01

    The camera is the only visible hardware; every wire is sewn into the fabric of the cap

    Why: A device that visibly marks its wearer as impaired is a device people stop wearing in public — so the cap had to look like a cap first.

  2. Decision 02

    Direction as a clock face, distance as beep frequency — not spoken coordinates

    Why: It is the same mental model canes and orientation training already use, so the device extends a skill the user already has instead of teaching a new one.

  3. Decision 03

    Two verbosity levels: object names only, or names with colour

    Why: 'Basic' is faster to parse while moving; 'detailed' helps when standing still and choosing between similar objects. The same feedback density isn't right for every moment.

Results

What actually happened

  • Submitted to the ETH Zurich Build2gether 2.0 Inclusive Innovation Challenge, alongside RescueTap
  • Recognised in the top 1% of 664 global participants

Photo

Youssef wearing the EchoGuide cap: a webcam clipped to the brim and over-ear headphones
Only the camera shows. The wiring is sewn into the fabric so the cap reads as a cap, not a medical device

For the technical reader

Technical implementation

The solution

How it works

A Raspberry Pi 4 wearable cap with a Logitech HD webcam sees the room in real time; a quantized TensorFlow Lite SSD MobileNet v1 model (trained on COCO classes) identifies objects in the frame; the result is spoken back through headphones. Direction is described using a clock-face convention — 'cup at two o'clock' — and distance is conveyed by beeps that speed up as the object gets closer. Two verbosity modes let the user choose between just object names or object-plus-colour, the latter using a KDTree nearest-match against the CSS3 colour table.

Stack

What, and why

TensorFlow Lite, SSD MobileNet v1 (quantized)
small and fast enough to run object detection on a Pi 4 in real time — a full-size model would be too slow to be useful while walking
OpenCV
video capture and frame handling, threaded separately from inference so audio feedback doesn't stall waiting on a frame
pyttsx3 + SpeechRecognition
on-device text-to-speech and voice input — no network round-trip, which matters when the device needs to work as reliably indoors as it does outside coverage
webcolors + scipy KDTree
nearest-neighbour colour naming against the CSS3 table, fast enough to run per-detection without noticeable lag

Implementation

A few details worth explaining

  • Video capture runs on its own thread so a slow inference pass never blocks the camera loop — a common failure mode in single-threaded CV demos that would have made the audio feedback feel laggy and unreliable
  • Object detection outputs are COCO class labels; for each detection I additionally sample the bounding-box region for a dominant colour and match it via KDTree against the CSS3 name table, so 'detailed' mode can say 'red cup' rather than just 'cup'

Challenges

What broke, and what I'd change

  • SSD MobileNet is fast but not always precise on partially occluded objects — in a cluttered room it sometimes named the wrong item in a cluster. A larger model would fix this at the cost of the real-time performance the whole device depends on; I chose speed, and it's the trade-off I'd revisit first with more compute budget
  • Getting the audio feedback timing right — frequent enough to be useful, sparse enough not to overwhelm — took more iteration than the computer vision did. This is a UX problem wearing a technical costume

What I learned

The hardest part of this project was never the model — it was deciding how much information a person can absorb through one sense while their attention is on walking safely. That's a product question, not a machine-learning question.

More work

Fintech

CIB Digital Banking

Payment features that clear compliance and ship to a national bank's customers.

  • Node.js
  • NestJS
  • REST API design
  • React
  • +3

Read case study

GlimpseGems home screen with search and mood categoriesGlimpseGems Explore screen with the list and map toggleGlimpseGems saved collection filtered by Favorites, To Visit and Visited
Mobile

GlimpseGems

A mood-based city discovery app, designed end to end — from first sign-in to a saved, shareable list.

Product design

  • Onboarding
  • Search & discovery
  • Map / list
  • Filtering
  • Saved states
  • Gamification

Read case study

Contact

Have a problem worth solving?

I'm open to Software Engineer and Product Engineer roles — remote, or relocating. Based in Cairo, EET (UTC+2).