atSSD: A Smartphone-Based Audiotactile Sensory Substitution Device for Hearing-Impaired Individuals

Abstract

By 2050, over 700 million people will experience disabling hearing loss. Cochlear implants remain limited by electrode count, frequency filtering, and high cost. This paper presents the atSSD (audiotactile sensory substitution device), a smartphone and ESP32-based wearable that converts environmental sound and spoken language into vibrotactile patterns on the upper body. A spectral pathway maps FFTderived frequency bands to a 3 × 3 motor matrix at approximately 62 ms latency. A semantic pathway translates recognized words into sign-language-derived vibration patterns with 144.4 ms internal latency. The system achieves an estimated battery life of 15.7 h, iOS speech-to-text accuracy of 98% in quiet conditions, and 125 predefined vibration patterns. A pilot study (n = 12) showed significant improvement in vibrotactile sound localization across sessions (p = 0.022). With material costs of 80-210e, the atSSD leverages existing smartphone infrastructure for low-cost audiotactile assistance.