Commit
•
56d37a4
1
Parent(s):
4c47348
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,7 +14,7 @@ tags:
|
|
| 14 |
- music ai
|
| 15 |
---
|
| 16 |
|
| 17 |
-
# Music Llama
|
| 18 |
|
| 19 |
## Llama for MIDI music generation
|
| 20 |
|
|
@@ -42,20 +42,21 @@ tags:
|
|
| 42 |
### Training Data
|
| 43 |
|
| 44 |
- [projectlosangeles/Monster-MIDI-Dataset](https://huggingface.co/datasets/projectlosangeles/Monster-MIDI-Dataset)
|
|
|
|
| 45 |
|
| 46 |
#### Training Hyperparameters
|
| 47 |
|
| 48 |
-
- config: tv2o-
|
| 49 |
- max-len: 4096
|
| 50 |
-
- layers:
|
| 51 |
- acc-grad: 8
|
| 52 |
- lr: 1e-4
|
| 53 |
- weight-decay: 0.01
|
| 54 |
- batch: 8x2x8
|
| 55 |
-
-
|
| 56 |
|
| 57 |
#### Loss
|
| 58 |
-
- train loss:
|
| 59 |
|
| 60 |
***
|
| 61 |
|
|
|
|
| 14 |
- music ai
|
| 15 |
---
|
| 16 |
|
| 17 |
+
# Music Llama Medium
|
| 18 |
|
| 19 |
## Llama for MIDI music generation
|
| 20 |
|
|
|
|
| 42 |
### Training Data
|
| 43 |
|
| 44 |
- [projectlosangeles/Monster-MIDI-Dataset](https://huggingface.co/datasets/projectlosangeles/Monster-MIDI-Dataset)
|
| 45 |
+
- [SymphonyNet MIDI Dataset](https://symphonynet.github.io/)
|
| 46 |
|
| 47 |
#### Training Hyperparameters
|
| 48 |
|
| 49 |
+
- config: tv2o-medium
|
| 50 |
- max-len: 4096
|
| 51 |
+
- layers: 12
|
| 52 |
- acc-grad: 8
|
| 53 |
- lr: 1e-4
|
| 54 |
- weight-decay: 0.01
|
| 55 |
- batch: 8x2x8
|
| 56 |
+
- bf16 mixed precision
|
| 57 |
|
| 58 |
#### Loss
|
| 59 |
+
- train loss: ...
|
| 60 |
|
| 61 |
***
|
| 62 |
|