|
--- |
|
license: mit |
|
--- |
|
|
|
This repo contains a low-rank adapter for LLaMA-7b finetuned on Ntropy proprietary dataset (consumer financial transactions). |
|
|
|
This version of the weights was trained with the following hyperparameters: |
|
|
|
- Base Model: decapoda-research/llama-7b-hf |
|
- Epochs: 10 (load from best epoch) |
|
- Batch size: 32 |
|
- Cutoff length: 1024 |
|
- Learning rate: 3e-4 |
|
- Lora r: 16 |
|
- Lora target modules: q_proj, k_proj, v_proj, o_proj |
|
|
|
Instructions for running it can be found at https://github.com/ntropy-network/enrichment_models/blob/main/notebooks/llama.ipynb |
