🎵 Wi-Fi Theremin: Sonification of CSI Data

[francescopace]

Overview

I've created an interactive web-based tool that transforms Wi-Fi Channel State Information (CSI) into sound, turning ESPectre into a musical instrument. The espectre-theremin.html file provides real-time audio sonification of movement detection data.

What It Does

The Wi-Fi Theremin converts CSI movement data and features into audio using the Web Audio API, creating an interactive experience that makes invisible Wi-Fi signals audible.

Core Functionality

Three Operating Modes:

• Continuous (Glissando): Smooth frequency transitions like a traditional theremin
• Quantized: Discrete musical notes for melodic output
• Hybrid: Combines both modes based on movement dynamics

Musical Scales:

• Pentatonic, Major, Minor, Chromatic
• Auto-scale selection based on signal complexity (entropy + variance)

Smooth Audio:

• 60Hz interpolation loop for smooth transitions between 1Hz MQTT updates
• Configurable smoothing factor for audio responsiveness

Feature-Based Modulation

The most interesting aspect is how all 10 CSI features modulate different audio parameters in real-time:

1. Waveform Type (entropy): Dynamically switches between sine/square/sawtooth
2. Vibrato (temporal_delta_mean + spatial_variance): Speed and depth modulation
3. Dynamic Filter (kurtosis + skewness): Low-pass cutoff and resonance
4. Stereo Pan (spatial_correlation): Spatial positioning in stereo field
5. Tremolo (temporal_delta_variance + spatial_gradient): Volume modulation
6. Auto Scale Selection (entropy + variance): Automatic musical scale selection

Each modulation can be individually enabled/disabled with configurable parameters.

Technical Implementation

• Web Audio API: Oscillator, gain, biquad filter, and stereo panner nodes
• MQTT WebSocket: Real-time data streaming from ESPectre
• Musical Note Detection: MIDI note conversion and display
• Single HTML file: No external dependencies, works offline

Use Cases

• Educational: Understand CSI features through sound
• Research: Data sonification for pattern recognition
• Interactive demos: Engaging way to demonstrate Wi-Fi sensing
• Musical performance: Actual playable instrument controlled by movement

Try It

1. Open espectre-theremin.html in your browser
2. Configure MQTT connection (same as espectre-monitor.html)
3. Click Connect
4. Move around and hear the Wi-Fi signals!