GestoLumina: Gesture interpreted Light, Sound and Haptics. Towards a Framework for Universal Music Design

Introducing GestoLumina (GeLu), a novel interface by the Universal Music Design (UMD) team at CymaSpace, whose mission is to enhance arts and culture accessibility for the Deaf and Hard of Hearing (DHH). UMD is a program that applies principals taken from the ”Universal Design” concept towards music accessibility and inclusion for the DHH. The primary goal of UMD is to create experiential, educational pathways for the DHH to interact with music in a meaningful way. GeLu is the first UMD interface in the framework and is modular in its feature set allowing the user to choose, based on use case, between visual and/or haptic feedback along with gestural sensing. GeLu combines two bracelets: a ring system with gesture sensing and haptic feedback; and an audio-reactive LED color feedback array. The ring system captures tactile input through sensors and provides haptic feedback on the fingertips. The audio-reactive LED color feedback bracelet visualizes audio data, creating dynamic visualizations synchronized with sound.