I wonder if this would be applicable to video game music. Be able to make stuff that's less repetitive but also smoothly transitions to specific things with in-game events.
This made me think about procedurally generated / data feed based game music.. how about never ending background music that's never (almost) the same than before?
Yeah, I agree. It's not going to replace modern music theory today or tomorrow, but I'm excited to play with this sort of thing in dynamic video game content generation. I'm envisioning a user giving a bit of information about their mood and then getting auto-generated music while they use an app briefly. What I heard here seems good enough to impress some people with that use case.
Indeed. Another fascinating approach to generative music is to code it out, and add some randomness, like this guy who does a pretty good stab at imitating Keith Jarrett: http://homepage.mac.com/digego/study_in_keith.mov (25MB mov file)
This really shouldn't have the problem of repetition. You even make events in the game affect the music, to make it less monotone, and add to the mood of the game. It has been tried, but as far as I know no one has actually published something like this yet.
This is missing the complexity to a lot of music. Swap "music" in your post for "books" or "software" or "human interactions" and maybe the skipped complexity becomes obvious. What if the key ingredient to someone's music is a long over-arching structure throughout the song of highs and lows following the emotional progression of a predictable 3-act story? The software would need to be capable of recognizing and synthesizing and tuning that kind of high-level element and surely many others like that. Even songs that are made up of a few repeating tracks overlaid probably have high-level qualities like that the artist tweaks the setup until the end results meet the high-level goal they were aiming for.
I'm sure a musician who was also a programmer who understood their own tastes and creation process well enough could currently create something like this for a specific genre, but I think we're very far from a one-size-fits-all generator.
It could also have some interesting avenues, like feeding some variables to the AI from say a video game (number and type of monsters on screen, mood etc.) to generate music reacting to what is happening on screen
I actually will like to see them do something novel and great with their technology.
As a developer, occasional FPS gamer, and musician, I'd like to see them tackle adaptive generative music that is actually convincing. I want music that takes cues from the gaming environment without obvious loop splicing points and without feeling mechanical.
To illustrate more this idea, let's use soundtrack v=negh-3hi1vE on youtube. Such soundtracks consist of multiple more or less repeating patterns. The period of each pattern is different: some background pattern that sets the mood of the music may have a long period of tens of seconds. The primary pattern that's playing right now has a short period of 0.25 seconds, plays for a few seconds and then fades off. The idea is to split the soundtrack into 10 second chunks and map each chunk to a vector of a fixed size, say 128. The same thing we do with words. Now we have a sequence of shape (?, 128) that can be theoretically fed into a music generator and as long as we can map such vectors back to 10 second sound chunks, we can generate music. Then we introduce a similar sequence that splits the soundtrack into 5 second chunks. Then another sequence for 2.5 seconds chunks and so on. Now we have multiple sequences that we can feed to the generator. Currently we take 1/48000th second slices and map them to vectors, but that's about as good as trying to generate meaningful text by drawing it pixel by pixel (which we can surely do and the model will have 250 billion weights and take 2 million years to train on commodity hardware).
Surprisingly good. Not Good good but I expected a lot worse.
I think there's great potential for that kind of music in video games, if they can procedurally generate it on the fly based on the current game state (think e.g. roguelikes).
When I read Hawkins's book _On Intelligence_ back in the mid-2000s I had thought it'd be cool to generate music by having the system predict what was novel and what was familiar. By mixing some novel and familiar notes/rhythm/tempo/timbre/percussion into the stream I should be able to make new music from scratch. I was annoyed that so many systems trained blindly on existing music instead of using first principles to generate something (although existing music seems useful for seeding the novelty/familiarity parameters). For games especially it would be nice to turn up or down different types of novelty to match what the player is doing.
However, as often happens, I got distracted before I got there. I learned Pure Data, and then got into audio synthesis, and then got into signal processing, and that led to procedural map generation … :)
If anyone's interested in more examples, I spent last month doing a "thing a day" bunch of procedural music experiments with Web Audio. A couple of the ones that seemed to work:
Also, "I want 'random' songs that are musically appealing to hear in sequence." A sort of lite DJ logic, really; not necessarily modifying the songs' entrances and exits, but more selecting keys and tempos.
I think we still need the step where the AI learns what a high quality sound library sounds like and then applies the previously learned abilities by triggering sounds of that library via MIDI.
That way you'd get perfect audio quality with the creativity of a musical AI.
Perhaps some kind of "generative listening" too, taking existing music and adapting it to the listener. Like how a lot of AI art is some kind of "remix" of existing things.
reply