README.md

---
base_model: unsloth/llama-3-8b-bnb-4bit
library_name: peft
license: mit
datasets:
- matiusX/legislacao-ufam
language:
- pt
- en
---

# Model Details

### Model Description

This model is a fine-tuned version of LLaMA 3 utilizing the Quantized Low-Rank Adaptation (QLoRA) technique. 
It is designed to answer questions related to the academic legislation of the Universidade Federal do Amazonas (UFAM). 
The training process involved generating a synthetic dataset of questions and answers based on the legislation, 
which includes various resolutions and norms provided by UFAM.


- **Developed by:** Matheus dos Santos Palheta
- **Model type:** More Information Needed
- **Language(s) (NLP):** Portuguese, English
- **License:** MIT
- **Finetuned from model:** unsloth/llama-3-8b-bnb-4bit

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]


## Uses

This model is intended for use by anyone with questions about UFAM's legislation. It is especially designed for students, professors, and administrative staff who need quick and accurate answers regarding academic policies and regulations. The model aims to support these groups by providing reliable information, thereby facilitating a better understanding of the rules and guidelines that govern their academic and professional activities at UFAM.
### Direct Use

This model can be directly used to answer questions regarding UFAM's academic legislation without additional fine-tuning.

### Downstream Use

The model can be integrated into larger ecosystems or applications, particularly those focusing on academic information systems,
legal information retrieval, or automated student support systems from UFAM.

### Out-of-Scope Use

This model is not suitable for general-purpose question answering beyond the scope of UFAM's academic legislation. 
It should not be used for legal advice or any critical decision-making processes outside its trained domain.

## Bias, Risks, and Limitations

While the model has been fine-tuned for accuracy in the context of UFAM's legislation, it may still exhibit biases present in the training data. 
Additionally, the model's performance is constrained by the quality and comprehensiveness of the synthetic dataset generated.

## How to Get Started with the Model

Use the code below to get started with the model.

```python
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps "xformers<0.0.27" "trl<0.9.0" peft accelerate bitsandbytes

from datasets import load_dataset
from datasets import Dataset
import pandas as pd

from unsloth import FastLanguageModel
import torch
max_seq_length = 2048 
dtype = None 
load_in_4bit = True 
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "matiusX/lamma-legis-ufam",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)
FastLanguageModel.for_inference(model)

prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

inputs = tokenizer(
[
    prompt.format(
        contexto, # contexto
        pergunta, # pergunta
        "", # resposta - deixar em branco
    )
], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)
tokenizer.batch_decode(outputs)

from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer, skip_prompt=True)
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 128)
```

## Training Details

### Training Data

The training data for this model is based on the academic legislation of UFAM. It includes a wide range of documents, 
such as resolutions and norms, which have been pre-processed and structured to create a synthetic dataset of questions and answers. 
For more details on the dataset, including the pre-processing and filtering steps, please refer to the Dataset Card available [here](https://huggingface.co/datasets/matiusX/legislacao-ufam).

### Training Procedure

#### Training Hyperparameters

- **Training regime:** Mixed precision (fp16)
- **LoRA configuration:**
  - **Alpha:** 16
  - **Dropout:** 0
  - **Target modules:** down_proj, up_proj, q_proj, gate_proj, v_proj, o_proj, k_proj

#### Speeds, Sizes, Times [optional]

- **Global Step:** 60
- **Metrics:**
  - **Train Runtime:** 1206.8508 seconds
  - **Train Samples per Second:** 0.398
  - **Train Steps per Second:** 0.05
  - **Total FLOPs:** 4.451323701362688e+16
  - **Train Loss:** 0.9744117197891077

![Alt Text](output.png)

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

[More Information Needed]

#### Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

### Results

[More Information Needed]

#### Summary



## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

[More Information Needed]

#### Software

[More Information Needed]

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]


## More Information [optional]

[More Information Needed]

## Model Card Contact

[email protected]

### Framework versions

- PEFT 0.12.0