Social XR: EEG-Driven Visualization in VR Using Multi-Headband EEG and Unsupervised ML
27th Annual Mersivity / WaterHCI Symposium, December 2025
Toronto and online, December 4 to 21, 2025 · Published in the symposium proceedings (open access on Zenodo)
The pipeline on screen: band powers from two headbands, clustered into low, medium and high activation and smoothed into a score.
Social VR gives a strong sense of being together but hides the moment-to-moment states that make face-to-face company work. We asked whether a consumer headband could add a conservative shared cue: not emotion recognition, just a 0 to 10 activation score each person can choose to share. Band powers from Muse S and Muse 2 headbands go through k-means and a Gaussian mixture to a smooth score that drives a Unity visualisation. This is the ancestor of the Friends feature in 6ense.
Abstract
Social VR/XR environments create a strong sense of shared presence, yet they often conceal the moment-to-moment internal states that support co-awareness in face-to-face interaction. This project investigates whether lightweight, consumer-grade EEG can be transformed into an interpretable and conservatively framed shared signal for Social XRβone that supports awareness and comparison without implying clinical-grade emotion recognition. We use Muse-class headbands and an end-to-end pipeline that combines signal processing, unsupervised learning, and Unity visualization. Our experiments use Muse S and Muse 2 headsets with four dry EEG channels (TP9, AF7, AF8, TP10) sampled at 256 Hz. From short windows, we compute baseline-normalized bandpower features and adopt a three-stage modeling narrative: (i) a transparent band-ratio activation heuristic to provide an interpretable ordering reference, (ii) KMeans clustering to obtain a coarse low/medium/high partition, and (iii) a diagonal-covariance Gaussian Mixture Model (GMM) initialized from KMeans to yield smooth, soft posteriors. We map these posteriors into a continuous 0β10 score that drives Social XR visualizations (avatar color and intensity). We also introduce MuseLog, a mobile app we developed to collect raw EEG from multiple Muse headbands concurrently, enabling synchronized multi-user datasets with reduced setup friction. In addition to raw EEG, MuseLog records relative band metrics, accelerometer and gyroscope signals, and per-device battery/status; the app is also designed to accommodate optical streams (e.g., fNIRS) for newer devices such as Muse S Athena. Screenshots of the multi-device workflow and a two-person conversation pilot highlight how low-friction multi-user acquisition can support Social XR research.
Authors
Alexander Vicol, Shubham Panchal, Steve Mann
Links
- Paper in the Mersivity / WaterHCI December-2025 Symposium Proceedings (Zenodo, open access)
- Related project: MuseLog
- Related project: Brain Data Visualization in VR/XR
- Related project: 6ense