Yeah, I agree. It's not going to replace modern music theory today or tomorrow, but I'm excited to play with this sort of thing in dynamic video game content generation. I'm envisioning a user giving a bit of information about their mood and then getting auto-generated music while they use an app briefly. What I heard here seems good enough to impress some people with that use case.
When I read Hawkins's book _On Intelligence_ back in the mid-2000s I had thought it'd be cool to generate music by having the system predict what was novel and what was familiar. By mixing some novel and familiar notes/rhythm/tempo/timbre/percussion into the stream I should be able to make new music from scratch. I was annoyed that so many systems trained blindly on existing music instead of using first principles to generate something (although existing music seems useful for seeding the novelty/familiarity parameters). For games especially it would be nice to turn up or down different types of novelty to match what the player is doing.
However, as often happens, I got distracted before I got there. I learned Pure Data, and then got into audio synthesis, and then got into signal processing, and that led to procedural map generation … :)
I imagine someday you'll be able to tie this to a neural sensor for a feedback loop that enables machine learning to generate the exact music for the exact mood you want.
Surprisingly good. Not Good good but I expected a lot worse.
I think there's great potential for that kind of music in video games, if they can procedurally generate it on the fly based on the current game state (think e.g. roguelikes).
I actually will like to see them do something novel and great with their technology.
As a developer, occasional FPS gamer, and musician, I'd like to see them tackle adaptive generative music that is actually convincing. I want music that takes cues from the gaming environment without obvious loop splicing points and without feeling mechanical.
Oddly, been mentioning this to students today - most of those I teach are completely unaware of such technology, or how it will impact them in the future. I think that in a few years time it'll be possible to generate a wide range of music at the touch of a button, and it'll be convincing.
This is clearly pastiche, but I think that's the first step towards convincing compositions and then onwards to being original and interesting. I know there have been software methods to compose (and symphonies written by such), but I think the current AI/machine learning trend will allow the technology to move far faster than teaching it explicitly all the music and composition theory that you can.
> I can't wait to hear some serious AI music-making a few years from now.
I think this will be particularly useful for musical compositions in movies and film, where the producer can "instruct" the AI about what to play, when, and how to transition so that the music matches the scene progression.
My only nitpick is I don't think you need AI or machine learning for automatic music generation. Instead, we've allowed tastes to be so dumbed down you could achieve success with MIDI, a RNG and a couple of if/then statements.
I wonder if this would be applicable to video game music. Be able to make stuff that's less repetitive but also smoothly transitions to specific things with in-game events.
It could also have some interesting avenues, like feeding some variables to the AI from say a video game (number and type of monsters on screen, mood etc.) to generate music reacting to what is happening on screen
There's a lot of negative comments here, but these are the earliest days and generating entire songs is kind of the hello world of this tech.
There's always going to be a balance between creating high level tools like this with no dials and low level tools with finer control, and while this touts itself as being "more controllable", it's clearly not there. But, the same way Adobe has integrated outpainting and generative fill into Photoshop, it's only a matter of time before products like this are built into Ableton and VSTs - where a creator can highlight a bar or two and ask your AI to make the the snippet more ethereal, create a bridge between the verse and the sax solo, or help you with an outro.
That said, similar to generating basic copy for a marketing site, these tools will be great for generating cheap background music but not much else, but any musician, marketing agency, or film-maker worth their salt is going to need very specifically branded music for their needs, and they're likely willing to pay for a real licence to something audiences will recognize, using generative AI and tools to remix the content to their specific need.
very cool. you'd think that with the web audio api there would have been an explosion of creative takes on music generation (like this one) but I am not sure it has happened...
I assume this is using a lot of soft synths and samples? Whatever they are, they sound really cheap. I hope I don't back myself into some Luddite corner here and this later turns out to be really impressive, like we are seeing in the image generation space. But given that we are living in a market of overabundance of stock photos, royalty free music, etc., I am sure that this is not endangering anything anytime soon.
And while comparing it to image generation, with Stable Diffusion and other models, a human has to be in the loop in order to generate the prompts, so we can't entirely replace them here either. How about an AI music generator that creates phrases, rhythms or sounds/VST presets based on a prompt for me?
If an average person can choose between listening to this, or listening to a musician that they have a personal connection to on their favorite streaming service, I wonder which one will be picked most of the time?
Meh. There’s plenty of deep learning music generation stuff out there, this is still a really cool approach
I think it would be cool to combine the two. Instead of generating raw midi, your GAN or reinforcement learning agent or whatever could try to generate sequences of transformations to melodic fragments.
Neural program synthesis type stuff.
Or maybe one could build an automatic music analysis tool that can start from the score and try to infer the program that generated them. (Is that a thing already?)
Is this targeting people who can already play instruments? I’d be interested in a raw audio based generation where I could interact with an RL agent by giving thumbs up/down to generated chunks. Does it make sense?
Yeah, it's only a matter of time before everyone has an AI composer on their phone continually creating works they are likely to enjoy. I wonder if it will feel formulaic.
As someone who's worked on this a lot, this article resonated but I disagree with the conclusions. (Or at any rate I feel like my procedural stuff is listenable, though I can't read or compose music.)
I think the author's biggest problem is the bottom-up approach of creating lots of raw clips and then at a higher level trying to assemble them. I've found music generation to consistently be a very similar task to text generation, and to me the author's approach seems analogous to generating lots of random sentences and then trying to assemble them into a short story.
Everything I've done that's worked has been completely top down - the core of the engine determines the state of the song (we're in the A section, bar 7, the current chord is V/V, etc.) and the low level parts generate something to match all that. E.g. the melody part might decide to play the root tone of the current chord, but it has to ask the core what note that would be.
The hard thing about this approach is that you need an environment where you can dynamically play notes of an arbitrary pitch/length/instrument, which either means lots of sound fonts or realtime audio gen. I've been doing this with WebAudio, and while it works it's been very painful - for me the "play sounds" part of procedural music has more difficult than the "compose music" part.
I'm very into both music composition/production and ML, so this is neat. That being said, I think the "computer generated music" path is probably the wrong approach in the short term. Music is really complex and leans heavily on both emotions and creativity, which aren't even on the radar for AI. Being able to dynamically remix and modulate existing music is still really cool though.
I would kill for a VST tool that would take a set of midi tracks, and synthesize a new track for a specific instrument that "blends" with them. I would also kill for something that can take a set of "target notes" and break them up/syncopate/add rests to produce good melodies, or take a base melody and suggest changes to spice it up.
This sounds pretty decent actually. I remember Stephen Wolfram's "A new kind of music" project which tried to generate music using cellular automata, with limited success.
reply