Menu
All projects

Research · Team project2025

DJESTHESIA

Tangible interaction for music and projected visuals.

Focus
Motion capture & interaction
My contribution
Interface testing & setup support

Team system

  • OptiTrack / Motive
  • ROS
  • UDP
  • JavaScript / WebGL

The team demo

8-second preview
DJESTHESIA team demo: overhead view of a DJ interacting with projected visuals

A short, muted excerpt of the team’s recorded demo. The full video is available on YouTube.

View team repositoryWatch team demo

The project

DJESTHESIA is a team research project exploring tangible interaction for DJs. It connects music, hand gestures and projected visuals, allowing performers to shape an audiovisual environment through physical interaction.

My contribution

I helped test the TouchDesigner–OptiTrack interface and supported the setup. The system and code shown here are the wider team’s work; the public repository documents a WebGL visualization pipeline.

How it connects

OptiTrack and Motive supply motion-capture data. ROS and UDP carry that information to a JavaScript/WebGL fluid simulation, whose visuals are projected onto the table. The repository also includes audio analysis and a bridge from tracked gestures to audio controls.

Team implementation and documentation:GitHubOfficial project page

Related publication

DJESTHESIA: Tangible Multimedia for DJs

SIGGRAPH Real-Time Live! · 10 August 2025

This publication is by Eduardo Castelló Ferrer. My contribution to the project was interface testing and setup support; I am not a listed author.

Read publication

System diagrams

Three views of the team’s system: interaction modes, audio routing and the visual pipeline. Open a diagram to inspect its labels.

System overview diagram with Performer, Gesture, Knob (Mixer), Visuals, Music and Performance, joined by four numbered interaction paths.View diagram ↗

Four interaction modes

I: mixer knobs change music. II: music changes visuals. III: gestures change visuals. IV: gestures change music. The diagram connects performer, interface, medium and performance.

Audio workflow diagram showing mixer and decks, four speakers, XLR, USB, a computer with audio software, FFT, Mouse Manager and Web Browser, and gesture-based data from ROS.View diagram ↗

Audio workflow

Mixer and decks connect to speakers through XLR and to a computer through USB. The computer contains Audio Composition Software, FFT, Mouse Manager and Web Browser, with gesture-based data arriving from ROS.

Visual workflow diagram labelled Sensing, Processing and Actuation, linking OptiTrack, Motive, ROS, UDP, Fluid Dynamics Simulation, HDMI and a projector.View diagram ↗

Visual information workflow

Three stages: Sensing with OptiTrack and Motive; Processing through ROS and UDP; Actuation through a fluid dynamics simulation, HDMI and a projector.

Diagram

Open original image