README.md

---
library_name: peft
thumbnail: https://www.superflows.ai/superflowsFaviconDark.png
base_model: HuggingFaceH4/zephyr-7b-beta
language:
- en
tags:
- endpoints_compatible
- inference
- text-generation-inference
inference: false
license: apache-2.0
pipeline_tag: text-generation
---

<a href="https://www.superflows.ai"><img src="https://raw.githubusercontent.com/Superflows-AI/superflows/main/public/sf-logo-long.png" alt="Superflows Logo" width="400" style="margin-left:'auto' margin-right:'auto' display:'block'"/></a>


# Model Card for Superflows 7B 1

This is a language model fine-tuned on data for calling APIs as functions, in the format used by [Superflows](https://github.com/Superflows-AI/superflows).

This model is trained from Zephyr 7B Beta (a chat-based finetune of Mistral 7B) using QLoRA, a parameter-efficient fine-tuning method.

Reach out to [Henry](mailto:[email protected]) if you're interested in self-hosting Superflows using this model.

## Model Details

### Model Description

- **Developed by:** [Superflows](https://github.com/Superflows-AI/superflows)
- **Model type:** 7B parameter GPT-like model fine-tuned using QLoRA on Superflows dataset
- **Language(s) (NLP):** Primarily English
- **License:** Apache 2.0
- **Finetuned from model:** [Zephyr 7B Beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)

## Uses

Use as the LLM for a Superflows AI Copilot embedded in a software product. The Copilot can call the software's APIs to answer user questions and complete tasks on behalf of the user.

### Out-of-Scope Use

Using it for anything other than as a Superflows AI Copilot for a software product.

## How to Get Started with the Model

Use the code below to get started with the model.

```
# Visit https://dashboard.superflows.ai to implement in your software product
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig

base_model_id = "HuggingFaceH4/zephyr-7b-beta"
ft_model_id = "Superflows/Superflows-1"

tokenizer = AutoTokenizer.from_pretrained(
    base_model_id, add_bos_token=True, trust_remote_code=True
)

config = PeftConfig.from_pretrained(ft_model_id)
model = AutoModelForCausalLM.from_pretrained(base_model_id)
ft_model = PeftModel.from_pretrained(model, ft_model_id)to("cuda")

# Example prompt
messages = [
  {"role": "system", "content": "You are Superflows chatbot AI. Your purpose is to assist users in Superflows via function calls\n\nSeek user assistance when necessary or more information is required\n\nAvoid directing users, instead complete tasks by outputting \"Commands\"\n\nToday's date is 2023-11-10.\n\nYou MUST exclusively use the functions listed below in the \"commands\" output. THIS IS VERY IMPORTANT! DO NOT FORGET THIS!\nThese are formatted with {{NAME}}: {{DESCRIPTION}}. PARAMETERS: {{PARAMETERS}}. Each parameter is formatted like: \"- {{NAME}} ({{DATATYPE}}: [{{POSSIBLE_VALUES}}]): {{DESCRIPTION}}. {{\"REQUIRED\" if parameter required}}\"\n1. list_users: List Users. PARAMETERS:\n- user_email (string): User's email address\n2. update_user: Update a User. PARAMETERS:\n- id (string): ID of user in UUID format. REQUIRED\n- email (string)\n- givenName (string)\n- familyName (string)\n- status (\"active\" | \"inactive\")\n\nTo use the output from a previous command in a later command, stop outputting commands - don't output the later command. If you output a command, you will be prompted again once it returns\n\nDon't copy the function outputs in full when explaining to the user, instead summarise it as concisely as you can - the user can ask follow-ups if they need more information\n\nAim to complete the task in the smallest number of steps possible. Be extremely concise in your responses\n\nThink and talk to the user in English\n\nThink step-by-step. Respond in the format below. Start with your reasoning, your plan, anything to tell the user, then any commands (you can call multiple, separate with a newline). Each section is optional - only output it if you need to. THIS IS VERY IMPORTANT! DO NOT FORGET THIS!\n\nReasoning: reason about how to achieve the user's request. Be concise. The user sees your reasoning as your 'thoughts'\n\nPlan:\n- short bulleted\n- list that conveys\n- long-term plan\n\nTell user: tell the user something. If you need to ask the user a question, do so here.\n\nCommands:\nFUNCTION_1(PARAM_1=VALUE_1, PARAM_2=VALUE_2, ...)\nFUNCTION_2(PARAM_3=VALUE_3 ...)"},
  {"role": "user", "content": "Update [email protected]'s email to [email protected]"}
]

prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

print(
    tokenizer.decode(
        ft_model.generate(**prompt, max_new_tokens=400, temperature=0.4)[0],
        skip_special_tokens=True,
    )
    .split("<|assistant|>")[-1]
    .strip()
)
```


## Training procedure


The following `bitsandbytes` quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: bfloat16

### Framework versions


- PEFT 0.7.0.dev0