The Flood Is Real. So Is the Chance to Rebuild. (2/3)

Inside the AI music boom: hype, power shifts, and what’s missing.

Part II: New Tools – How AI Is Reshaping the Music-Making Process

Part 2

In part 1 we’ve seen how fast generative music platforms are spreading, the next question is: What exactly are people doing with them? This section looks at the creative side—how AI is changing the process of making music. From AI-powered vocal cloning to real-time accompaniment and audio extension, we explore how these tools are reshaping not just what gets made, but how musicians think and work. The creative possibilities are vast—but they also raise urgent legal and ethical questions, which we’ll examine in the next part.

New Creative Tools and Possibilities Enabled by AI

Beyond generating songs from scratch, AI is expanding the creative toolkit for musicians and producers in powerful ways. Six innovations in particular are opening new frontiers for music creation:

• Stem Separation (Source Isolation): AI-driven stem separation algorithms can take a finished mixed song and split it into individual tracks or “stems” (vocals, drums, guitar, bass, etc.). This capability, pioneered by models like Deezer’s Spleeter, has revolutionized remixes and sampling. Artists can now isolate a cappella vocals from any recording or extract a specific instrument performance with a few clicks. This was famously used by the production team behind The Beatles to create a “final” Beatles song in 2023 – engineers employed AI-powered stem separation to isolate John Lennon’s voice from a lo-fi 1970s cassette demo, enabling a new studio arrangement around his vocals. For musicians, such tools mean new creative freedom: you can restructure classic tracks, mashup elements from different songs, or modernize old recordings by replacing instruments. Stem separation also aids live performances (e.g. generating backing tracks) and helps train other AI models by providing clean instrument examples. It’s a prime example of AI augmenting the music creation process rather than replacing it.
• Timbre Transfer (Audio Style Transfer): Timbre transfer applies the concept of style transfer (popular in visual art) to sound. In practice, it means using AI to re-render one audio signal with the sonic characteristics of another. For example, you could hum a melody and have it come out sounding like a violin, or take a piano piece and transform it so that it’s played by an electric guitar. Google’s Magenta team demonstrated this with their Tone Transfer project, where a user can sing or input any sound and the AI model outputs the same performance on a chosen instrument (trumpet, flute, etc.). This opens up novel artistic avenues: a composer without access to a rare instrument can simulate its sound with their voice; producers can morph vocals into synthesized pads or turn a simple whistle into a lush saxophone line. Timbre transfer essentially allows creative cross-pollination between instruments and even between human and non-human sounds. It lowers the barrier for producers to experiment with exotic sounds and gives artists new ways to express ideas (imagine beatboxing a drum line and converting it into a realistic drumkit recording). As these models improve, timbre transfer could become a staple in studios for crafting unique textures.
• Voice Cloning and AI Vocalists: Perhaps the most headline-grabbing innovation is AI voice cloning – the ability to closely mimic a singer’s voice with a model. Modern AI can learn a vocalist’s distinctive timbre and inflection from audio data, then generate new vocals saying (or singing) any lyrics in that voice. This has led to both creative experimentation and ethical debates. On one hand, artists like Grimes have embraced the technology: in 2023 she offered 50% royalties to anyone who creates an AI-generated song using her voice, actively encouraging fans to experiment. Avant-garde musician Holly Herndon similarly released Holly+, a free AI model of her voice that others can use in their music. These cases show a potential new model where artists “license out” their vocal likeness as a creative asset. On the other hand, unauthorized voice cloning has sparked backlash. An anonymous producer called Ghostwriter scored viral hits in 2023 by releasing songs with AI-generated vocals imitating stars like Drake and The Weeknd – without their consent. The track “heart on my sleeve” drew millions of streams before being removed, and it alarmed the industry about digital replicas of artists. Even legendary producer Timbaland briefly previewed a track featuring a deepfaked Notorious B.I.G. verse, prompting debate about the ethics of resurrecting deceased artists’ voices. In response, record labels and platforms have been quick to ban or penalize impersonations. Still, from a pure creativity standpoint, AI vocals allow for things like synthesizing backing vocals in an artist’s own voice, trying out song ideas with a “virtual singer,” or multilingual vocals without a human singer. They can also help producers demo a song in the style of a particular artist. The technology is advancing fast – new AI singers can capture not just tone but emotional expression and vibrato – raising the very real question of what constitutes a “human performance.” Voice cloning exemplifies the double-edged nature of AI in music: it unlocks thrilling creative possibilities, but also raises ethical and legal dilemmas about identity and consent.
• AI for Early Inspiration and Idea Generation: Musicians are increasingly using AI tools to kickstart their creative process—whether to spark song ideas, sketch melodies, or explore genre blends when inspiration runs dry. These systems can generate musical material from scratch, offering a starting point for further human refinement. Apps like Boomy allow anyone to create full songs in seconds by selecting a style or entering a prompt. Many users treat these tracks as rough drafts or mood sketches. Magenta Studio, a suite of AI plugins for Ableton Live, generates original MIDI melodies, drum patterns, and variations on user input. Producers often use its “Generate” function to overcome writer’s block by introducing fresh 4-bar ideas. One electronic musician described the tool as making composition “more intuitive and expansive,” enabling fast experimentation with motifs. Similarly, Orb Producer Suite suggests chord progressions, basslines, and melodies—serving as an inspiration engine for both beginners and experienced producers. These generative platforms also support exploration of hybrid genres. The public tool Suno encourages genre mashups; users can input combinations like “midwest emo with neo-soul” or “EDM with folk,” and the system returns a reference track in that style. This lets musicians imagine new sonic possibilities without needing to master each genre themselves. Beyond music generation, some artists use AI text models for songwriting—generating lyrics, titles, or thematic prompts—then pair those with AI-composed instrumentals. Together, these tools expand the creative palette and help artists move from blank page to prototype faster.
• AI for Audio Inpainting and Track Extension: One of the most promising uses of generative AI in music is audio inpainting—the ability to fill in gaps or extend recordings in a musically coherent way. Acting like a virtual collaborator, AI can take a fragment of a song and develop it into a complete piece. Tools like Udio offer an Extend function that allows users to upload a short audio clip—such as a loop, chorus, or stem—and generate additional sections to expand it. A 30-second sketch can become a 3-minute draft in minutes, cutting down hours of manual work. AI doesn’t just add material to the end; it can also regenerate sections within a track. With Udio’s Inpaint feature, creators can highlight a part of the song—an empty bridge, a flawed verse—and have the system rewrite that portion while leaving the rest untouched. Early adopters see these tools as a potential “killer app” for music AI. The ability to upload unfinished material and have the system develop it further has unlocked new workflows. Musicians are using AI to revive abandoned sketches, extend loops, or repair incomplete tracks—transforming fragments into listenable, structured compositions.
• AI-Generated Accompaniment and Backing Tracks: When a songwriter has a melody or chord progression but no full arrangement, AI tools can help by generating supporting parts—basslines, pads, counter-melodies—that fit the musical context. These systems act as virtual collaborators, offering quick ideas to flesh out a track. Some tools can even jam in real time. AI Duet, an experiment by Google, allowed users to improvise on piano while the AI responded live. Though basic, it demonstrated the potential for interactive accompaniment. More advanced research points to richer results: Sony CSL’s Diff-A-Riff model can generate “high-quality instrumental accompaniments that seamlessly integrate with a given musical context.” Still in development, it suggests a near-future scenario where a musician might play a few chords and ask: “AI, give me a reggae bassline and a jazz guitar backing.” The system could then return a realistic arrangement on demand. These tools could significantly accelerate the sketching phase of songwriting, making it easier to explore arrangement ideas without needing a full band or DAW expertise.

We’ve entered an era where creativity is no longer limited by access to instruments, studios, or even human vocal cords. These tools extend what’s creatively possible, especially for those outside traditional music infrastructures. But with this freedom comes ambiguity: Who owns the output? Who gets credited? As we move into the final part 3, we’ll look at the legal frameworks—or lack thereof—surrounding AI music. Because the tools might be new, but the need for attribution, rights, and compensation is as old as the music industry itself.