← All research projects

Neuromatic: A Multimodal Interface for Mobility Aid Using EEG and Gesture Recognition

27th Annual Mersivity / WaterHCI Symposium, Summer 2025
Toronto and online, June 20 to August 25, 2025 · Published in the symposium proceedings (open access on Zenodo)

A Raspberry Pi Zero board in the bottom half of a small printed case

The clip-on module's brain: a Raspberry Pi Zero 2W in its printed case, with a camera that attaches to the front of the Muse headband.

Neuromatic fuses two signals: brainwaves from a Muse S headband and hand gestures seen by a tiny camera clipped to the headband. Point your head at a device, the camera reads its QR code, and a gesture plus your EEG confirm the command. The idea is closed-loop control of the things around you for people with mobility impairments, with the processing done at the edge.

Abstract

Neuromatic is a wearable, multimodal interface that leverages brainwave data and computer vision to provide intuitive, reliable actuation of smart devices. By fusing EEG signals from the Muse S headband with real-time gesture recognition via a wearable Pi Camera module, the system provides a closed-loop control experience for users with mobility impairments. This paper presents the hardware design, communication protocols, and edge inference architecture used to implement Neuromatic.

Authors

Harrison Ossias, Eric Fan, Jessy Ma

Links

← All research projects