File size: 3,162 Bytes
2037f28 ce1ce01 2de6caa 4914c31 ce1ce01 72ef83d ce1ce01 ffc0f0a ce1ce01 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: mit
---
---
license: mit
---
# Model Card for shijunju/gemma_7b_finRisk_r10_4VersionQ
The repository includes only a LoRA adapter trained with keras-nlp on a TPU. To use the fine-tuned model, load Gemma-7b and apply `load_lora_weights` before generating output (see instructions below).
This model is fine-tuned using the LoRA (Low-Rank Adaptation) approach, specifically designed for question answering in the domain of financial risk compliance.
The Gemma-7b-en model is fine-tuned using documents from fincen.gov.
It is capable of answering questions about documents published on fincen.gov, including Alerts, Advisories, and Financial Trend Analysis reports since 2020.
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
- The model is created as part of experiment to find better models, a more accurate (70%-78%) finetuned model can found at: [shijunju/gemma_7b_finRisk_r6_4VersionQ](https://huggingface.co/shijunju/gemma_7b_finRisk_r6_4VersionQ)
- **Developed by:** Shijun Ju
- **Finetuned from model:** Gemma-7b-en
- The model is finetuned using Keras_nlp with TPU adapting the approach used at [https://www.kaggle.com/code/nilaychauhan/keras-gemma-distributed-finetuning-and-inference](https://www.kaggle.com/code/nilaychauhan/keras-gemma-distributed-finetuning-and-inference)
- LoRA rank: 10
- Accuracy rate for unseen paraphrased questions is 67% for default setting and 73% using Beam Search of 3.
### Dataset Used
[shijunju/fincen_all_questions_5versions](https://huggingface.co/datasets/shijunju/fincen_all_questions_5versions)
## How to Get Started with the Model
Use the code below to get started with the model. (**Require TPU!**)
```python
# https://www.kaggle.com/code/nilaychauhan/keras-gemma-distributed-finetuning-and-inference
import tensorflow_text as text
model_parallel = keras.distribution.ModelParallel(
device_mesh=device_mesh,
layout_map=layout_map,
batch_dim_name="batch"
)
keras.distribution.set_distribution(model_parallel)
gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma_7b_en")
gemma_lm.backbone.enable_lora(rank=10)
gemma_lm.backbone.load_lora_weights(MODEL_LORA_WT_PATH) # the folder and file name you save the downloaded adaptor from this repository
def generate_response(prompt, max_length=256):
outputs = gemma_lm.generate(prompt,
max_length=max_length
)
print(outputs)
inference_template = """<start_of_turn>user\nQuestion: {question}\n<end_of_turn>\n\n<start_of_turn>model\n"""
prompt = inference_template.format(
question="Identify the specific fraudulent scheme highlighted in the FinCEN alert related to requests for convertible virtual currency payments.",
response=""
)
print(generate_response(prompt))
```
If you encounter any errors, please refer to [https://www.kaggle.com/code/nilaychauhan/keras-gemma-distributed-finetuning-and-inference](https://www.kaggle.com/code/nilaychauhan/keras-gemma-distributed-finetuning-and-inference) for how to load the model into TPU properly.
## Model Card Contact
[email protected] |