I think a good shorthand for “slop” is that you spend less time creating something than it takes for someone to consume it.
What I love about live generation and performance is that it naturally balances the two through the helpful bottleneck on shared human experience.
One of the most fun things about latent spaces, is that they are actual spaces. Take a ride in you own spaceship to explore the constellations of latent music! magenta.withgoogle.com/spacedj-anno...
Powered by the Lyria RealTime API in AI Studio.
Thanks again to @hollyherndon.bsky.social, it's always fun collaborating with you 🙏
These latest features take us one step closer to live interactive generative music models that are open, personalizable, and run locally on-device, so you can really trust them to hack them like any other instrument.
Next step, live on-device inference, stay tuned… 😀
As we’ve seen with Genie, real-time interactive generative models opens up new worlds, and we’re excited to keep pushing the boundaries of what’s possible.
This new wave of features is the culmination of effort from many folks at Google DeepMind, on the Lyria team and beyond. We are grateful to work with so many amazing collaborators to build the next generation of human-centered music AI tools. 🙏
We hope Magenta RT finetuning can serve that purpose, and are happy to have their model as a first example of personalizing live generative music.
For years, Holly and Mat Dryhurst have pioneered artist-centric ways in which AI can be used to push the boundaries of expression, and how artists and communities can have more agency over developing models for their own purposes (as they open sourced her voice to holly.plus for experimentation).
We're thrilled to have worked with Holly Herndon to train a model finetuned on her vocal style to share with all of you.
Playing with Holly’s model, you can use additional controls to get vocalizations that contour the music or "color" the instruments with a timbre that resembles Holly's voice
We’re also excited to add support for artists to customize Magenta RT on their own training data!
g.co/magenta/rt-finetuning-colab
Audio injection also supports *live* music input. You can jam along with it and use it as an intelligent live looper!
Try it out for yourself on free Colab TPUs!
g.co/magenta/rt-audio-injection-colab
Magenta RT now supports what we call *audio injection* - a novel method for real-time audio input
You can use audio injection to transform and add layers to prerecorded music!
We’ve since been hard at work preparing new features. Check out our tech report on *Live Music Models*, including both Lyria RT (API) and Magenta RT (open)
📜 Paper: arxiv.org/abs/2508.046...
🔊Samples: g.co/magenta/live...
ICYMI, in June we shared a preview of Magenta RealTime - the first open weights model capable of *live* music generation
🤗 Model: huggingface.co/google/magen...
⭐ Code: github.com/magenta/mage...
Realtime interactive generative models FTW!
Announcing a new 🌊 of details and features for Magenta RealTime, the open weights live music AI model from GDM!
* Live Jamming with audio input 🎤🎸🎵
* Personalize your own models 🔧
* Tech report 📜
Links below in the 🧵...
Looking forward to seeing all sorts of interesting ways you mangle and twist the outputs into unique creations 🙂.
We are excited to make these models more accessible, and more tightly integrated into musician’s workflows as it will open up more opportunities to customize and personalize how you use them to come up with truly personal expression.
Built on the hard work of the Lyria Team, the VST plugin has been a labor of love of Ashutosh Desai. Thanks so much also to the support of artists Sidecar Tommy from Beats Antique, Toro y Moi in exploring the possibilities of DAW integration of our models.
The Infinite Crate is cross-platform, and currently supports Mac VST/AU, Windows VST, and a Mac Standalone application (no VST host required).
It is built on the Lyria RealTime API and thus requires an internet connection and API key (usage is free).
Be a Prompt DJ 🎧🎶: Play it like a live instrument. Use prompts and advanced controls to vary the instrumentation, texture, and genre. Incorporate with a DJ set to apply effects and mix with other loops.
Jam Along 🎸🥁: Create dynamic backing tracks that are different every time, varying and evolving as you play on top.
What can you do with it?
Dig in the crate 🎛️💡 : Explore new sounds by blending and merging genres, moods, instruments, or anything you can think of. Sample the stream of music to spark your own inspiration or help you get past a creative block.
New VST/AU Plugin! 🚨
Play with Lyria RealTime directly from inside your favorite DAW with “The Infinite Crate” 🎧🎶
With a VST plugin you can feed audio directly into your DAW for sampling, live performance, or even a practice partner to jam with.
💾 Get it here: g.co/magenta/infinite-crate
Try it out and fork it here:
bit.ly/3ZOp7KS
Don't forget you can type in new prompts,
Curious to see what else you can make :).
Pleasantly surprised with how easy/fun it is to "vibe code" new musical experiences in AI Studio.
I'm not much of a JS guy, but had Gemini help me make a "Kaoss Pad" emulator using the Lyria RealTime API to explore musical latent spaces in realtime.
How are they being impacted by the current approach to deploying/integrating AI in our societal habitat? How could we be more sustainably approaching things?
There's a lot of inspiration to be found in nature, seeing the interconnectedness of a habitat and the effects of disruption on one part of it. What are the core pillars of our societal habitat that we need to make sure survive going forward? How do support the broader system?
It brings up interesting questions. What are the things most essential that we want to preserve? How can we adapt them to help survive AI? How can we co-adapt AI to sustainably become a better part of the habitat? What inspiration and cautionary lessons can we learn from history?
Ecology isn't just about preserving by removing humans, it's about understanding how an environment responds to different impacts and thinking of interventions to help it adapt (+tech co-adapt) such as to not lose the things we find precious and essential to future thriving.
Lately I've been finding it interesting to think of AI from an ecology/conservationist perspective.
In the analogy, we are the native habitat (our society, culture, lives) and AI is the outside disruptive influence (i.e. invasive species, pollution, human effects, etc.)
