Hi everyone, I've been working on a Neural Network model that takes any mp3 as input and turns it into beatmaps for rhythm games. I have started with OSU as it is one of my favorites, but the long term goal is to support all kinds of different platforms:
- Beatmaps are converted to an intermediate file format, which is just an excel sheet containing "tracks" with timing. Only beat and hold events are encoded (OSU maps have 1 track, MANIA tracks have 4).
- Then this intermediate file format is tokenized (turned into numbers the model can understand), and is fed into the network combined with the audio.
- The model is then trained to predict the next token (a series of beatmap events) based on both previous events and the surrounding audio.
I am currently in the process of training the first models and making them publicly available:
https://github.com/sedthh/BeatLearning
- Beatmaps are converted to an intermediate file format, which is just an excel sheet containing "tracks" with timing. Only beat and hold events are encoded (OSU maps have 1 track, MANIA tracks have 4).
- Then this intermediate file format is tokenized (turned into numbers the model can understand), and is fed into the network combined with the audio.
- The model is then trained to predict the next token (a series of beatmap events) based on both previous events and the surrounding audio.
I am currently in the process of training the first models and making them publicly available:
https://github.com/sedthh/BeatLearning
