← All research projects

Social XR: EEG-Driven Visualization in VR Using Multi-Headband EEG and Unsupervised ML

27th Annual Mersivity / WaterHCI Symposium, December 2025
Toronto and online, December 4 to 21, 2025 · Published in the symposium proceedings (open access on Zenodo)

A lab with two large screens showing EEG frequency bands and scatter plots, people working at tables behind

The pipeline on screen: band powers from two headbands, clustered into low, medium and high activation and smoothed into a score.

Social VR gives a strong sense of being together but hides the moment-to-moment states that make face-to-face company work. We asked whether a consumer headband could add a conservative shared cue: not emotion recognition, just a 0 to 10 activation score each person can choose to share. Band powers from Muse S and Muse 2 headbands go through k-means and a Gaussian mixture to a smooth score that drives a Unity visualisation. This is the ancestor of the Friends feature in 6ense.

Abstract

Social VR/XR environments create a strong sense of shared presence, yet they often conceal the moment-to-moment internal states that support co-awareness in face-to-face interaction. This project investigates whether lightweight, consumer-grade EEG can be transformed into an interpretable and conservatively framed shared signal for Social XRβ€”one that supports awareness and comparison without implying clinical-grade emotion recognition. We use Muse-class headbands and an end-to-end pipeline that combines signal processing, unsupervised learning, and Unity visualization. Our experiments use Muse S and Muse 2 headsets with four dry EEG channels (TP9, AF7, AF8, TP10) sampled at 256 Hz. From short windows, we compute baseline-normalized bandpower features and adopt a three-stage modeling narrative: (i) a transparent band-ratio activation heuristic to provide an interpretable ordering reference, (ii) KMeans clustering to obtain a coarse low/medium/high partition, and (iii) a diagonal-covariance Gaussian Mixture Model (GMM) initialized from KMeans to yield smooth, soft posteriors. We map these posteriors into a continuous 0–10 score that drives Social XR visualizations (avatar color and intensity). We also introduce MuseLog, a mobile app we developed to collect raw EEG from multiple Muse headbands concurrently, enabling synchronized multi-user datasets with reduced setup friction. In addition to raw EEG, MuseLog records relative band metrics, accelerometer and gyroscope signals, and per-device battery/status; the app is also designed to accommodate optical streams (e.g., fNIRS) for newer devices such as Muse S Athena. Screenshots of the multi-device workflow and a two-person conversation pilot highlight how low-friction multi-user acquisition can support Social XR research.

Authors

Alexander Vicol, Shubham Panchal, Steve Mann

Links

← All research projects