Abstract
This paper describes a real-time, multi-user music generation platform integrated into a video sharing platform, where many viewers jointly co-create a single evolving musical track during a live session. On such platforms viewers are typically passive consumers, and traditional digital audio workstations impose a high barrier to collaborative music-making. A generative music model synthesizes a cohesive track from simultaneous, heterogeneous inputs, including text prompts, hummed melodies, and Musical Instrument Digital Interface (MIDI) data. A harmonic consensus stage projects these inputs into a shared harmonic latent space, resolving conflicts among competing inputs into a single harmonic guide that fixes a unified key, tempo, and chord progression. Conditioned on this guide, separate instrument stems for drums, bass, synth, and vocals are generated in parallel and mixed in real-time into one audio stream. An overlay on the video player visualizes each contribution and exposes controls through which viewers vote on musical direction, adjust individual stem volumes, and add layers, recasting passive viewers as active co-creators.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Shin, D., "Real-Time Collaborative Generative Music Jamming on a Video Sharing Platform", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/11270