Submersivity: XR-Guided Garbage Detection with an RC Submarine for Peter Street Basin Cleanup
IEEE International Conference on Mechatronics and Automation (ICMA) 2026
Changchun, China, August 2026 · Published; full paper on IEEE Xplore
The pipeline, from the paper: the ESP-DIVE submarine captures underwater video, a fine-tuned YOLOv8s detector finds debris, detections are placed on a photogrammetry model of the basin, and Vuzix Smart Swim goggles show distance and direction to the nearest target.
Peter Street Basin, the 60 m by 30 m slip on Toronto’s waterfront that our club cleans every week, collects litter that is hard to see from the surface. Submersivity sends a battery-powered RC submarine (the open-source ESP-DIVE with a XIAO ESP32-S3 Sense camera) under the water and runs its video through a YOLOv8s detector fine-tuned on 12,177 terrestrial and underwater debris images.
Each detection is placed on a map of the basin built with structure-from-motion photogrammetry and ArUco markers, then sent to Vuzix Smart Swim XR goggles as distance, direction and class cues, so a swimmer can go straight to the nearest piece of garbage. Early field results show detections holding up in the basin’s poor visibility.
Abstract
This paper presents an underwater garbage detection and visualization system that combines a remote-controlled submarine, real-time computer vision, basin-scale localization, and an extended reality (XR) head-up display (HUD). A battery-powered submersible based on the open-source ESP-DIVE platform, equipped with a XIAO ESP32-S3 Sense camera, captures underwater imagery and transmits frames for inference using YOLOv8s fine-tuned on a combined terrestrial and underwater debris dataset of 12,177 images. Detected objects are localized within a mapped model of Peter Street Basin (PSB) using structure-from-motion photogrammetry (COLMAP) and ArUco marker registration, then transmitted to Vuzix Smart Swim XR goggles as distance, direction, and class cues for guided cleanup. The purpose of the system is to make underwater litter survey and recovery more interactive, more intuitive, and more spatially aware. In addition to live detections, the user interface displays a reduced map of PSB with accumulated garbage locations and highlights the nearest target using a compass-style indicator. Preliminary field results demonstrate successful detection of underwater debris under challenging visibility conditions, along with real-time visualization of target direction and range on the XR display.
Authors
Chris McGale, Michel A. Herrera Viyella, Chan Park, Daniel Bros, Alexander Vicol, Steve Mann
Links
- Paper on IEEE Xplore (DOI 10.1109/ICMA69663.2026.11647510)
- Related project: Telekolympographic Swim Control
- Related project: Swiminalization and (K)No(t)-Swimming