Neuromatic: A Multimodal Interface for Mobility Aid Using EEG and Gesture Recognition
27th Annual Mersivity / WaterHCI Symposium, Summer 2025
Toronto and online, June 20 to August 25, 2025 · Published in the symposium proceedings (open access on Zenodo)
The clip-on module's brain: a Raspberry Pi Zero 2W in its printed case, with a camera that attaches to the front of the Muse headband.
Neuromatic fuses two signals: brainwaves from a Muse S headband and hand gestures seen by a tiny camera clipped to the headband. Point your head at a device, the camera reads its QR code, and a gesture plus your EEG confirm the command. The idea is closed-loop control of the things around you for people with mobility impairments, with the processing done at the edge.
Abstract
Neuromatic is a wearable, multimodal interface that leverages brainwave data and computer vision to provide intuitive, reliable actuation of smart devices. By fusing EEG signals from the Muse S headband with real-time gesture recognition via a wearable Pi Camera module, the system provides a closed-loop control experience for users with mobility impairments. This paper presents the hardware design, communication protocols, and edge inference architecture used to implement Neuromatic.
Authors
Harrison Ossias, Eric Fan, Jessy Ma
Links
- Paper in the Mersivity / WaterHCI Summer-2025 Symposium Proceedings (Zenodo, open access)
- Related project: Mind over Motor
- Related project: EEG Word Keyboard
- Related project: TrembleCV