---
license: apache-2.0
pipeline_tag: text-generation
tags:
  - finetuned
inference:
    parameters:
        temperature: 0.4
---

## Preamble
The main purpose of that model, outside form providing a strong foundation for assisted prompting, was to better understand how fine-tuning works.
Therefore, the dataset is prone to change, as well as the training workflow.

---

# IMAGINE-7B-Instruct

> *Interaction Model for Advanced Graphics Inference and Exploration*

This Large Language Model (LLM) is a fine-tuned version of [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1). It is designed to integrate the conversational method into the process of generating image prompts.
This model excels in understanding and responding to prompts related to image generation through an interactive dialogue. This innovative approach allows users to engage in dialogues, providing textual prompts that guide the model in generating corresponding sets of tokens. These tokens, in turn, serve as dynamic prompts for subsequent interactions.

**IMAGINE** enhances the user experience by seamlessly converting visual ideas into a format that can be further utilised or interactively refined within a text-based conversational context.


# Model Details

# Model Description
- **Developed by**: [Syntonomous Lab](https://syntonomous.com)
- **Model type**: text-generation
- **Parent Model**: [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)

## Model Architecture
This instruction model is based on [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1), a transformer model with the following architecture choices:
- Grouped-Query Attention
- Sliding-Window Attention
- Byte-fallback BPE tokenizer

# 💻 Get Started with IMAGINE

## Prompt template
To leverage instruction fine-tuning, your prompt should be surrounded with `[INST]` and `[/INST]`.

```text
<s>[INST] {your prompt goes here} [/INST]
```

## Instruction format

Here is a basic example of how to use **IMAGINE-7B-Instruct** using Mistral's instruction format.


```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

MODEL_NAME = "syntonomous/IMAGINE-7B-Instruct"
model = AutoModelForCausalLM.from_pretrained(MODEL_NAME)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"

prompt = "<s>[INST] Help me create the prompt to generate an image that capture an intense moment of life [/INST]"


pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer
)

generated = pipe(
    prompt,
    do_sample=True,
    temperature=0.4,
    pad_token_id=tokenizer.eos_token_id,
    max_new_tokens=1000
)

print(generated[0]["generated_text"].split("[/INST]")[1].strip())
```

# Training Details

## Data
The dataset used to fine-tune this model has been entirely created by [Syntonomous](https://syntonomous.com) and does not contain any external sources. For more information on how the original [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) was fine-tuned, please refer to their model page.

## Compute Infrastructure
- **Hardware**: 
  - 4x Nvidia Tesla V100S 32GiB
  - 160 GiB RAM
  - 52 vCores CPU
- **Compute Region**: Europe
- **Training Effective Duration**: 6 hours/resource (=24 hours)
- **Carbon Emitted**: 0.72kg CO² *(300W * 24h = 7.2 kWh * 0.1kg CO²)*

---

*✦ Syntonomous*\
Learn to share — Share to learn