step650 model was trained in 48hr on an RTX3090, step1100 model initially experimentally trained to add tiny attenion (RWKV-4b), but then finetuned again to remove it. Final finetuning was done with ctx 4096.
Trained using GiantMIDI-Piano dataset, augmented 30x. See training history: https://wandb.ai/brianflakes/rwkv-midi
To run, use the rwkv pip package: https://pypi.org/project/rwkv/
To convert outputs to MIDI, use https://github.com/briansemrau/MIDI-LLM-tokenizer