---
license: other
inference: false
---
**Sailor 1.8B AWQ**

- Model creator: Sea AI Lab
- Original model: Sailor 1.8B

Sailor is a suite of Open Language Models tailored for South-East Asia (SEA), focusing on languages such as 🇮🇩Indonesian, 🇹🇭Thai, 🇻🇳Vietnamese, 🇲🇾Malay, and 🇱🇦Lao. Developed with careful data curation, Sailor models are designed to understand and generate text across diverse linguistic landscapes of SEA region. Built from Qwen 1.5 , Sailor encompasses models of varying sizes, spanning from 0.5B to 7B versions for different requirements. We further fine-tune the base model with open-source datasets to get instruction-tuned models, namedly Sailor-Chat. Benchmarking results demonstrate Sailor's proficiency in tasks such as question answering, commonsense reasoning, and other tasks in SEA languages.

**Description**

This repo contain AWQ format model files for Sailor Sailor 1.8B.

**Prompt Format**

```
prompt_template = "{prompt}"
```

**Quickstart**

Here provides a code snippet to show you how to load the tokenizer and model and how to generate contents.

- Using transformers

```
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Matheusuz/Sailor-1.8B-AWQ"

# Model
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    low_cpu_mem_usage=True,
    device_map="cuda:0"
)

# Tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Prompt template
prompt_template = "Artificial intelligence is"

# Convert prompt to tokens
tokens = tokenizer(
    prompt_template,
    return_tensors='pt'
).input_ids.cuda()

# Model parameters
generation_params = {
    "do_sample": True,
    "temperature": 0.7,
    "top_p": 0.95,
    "top_k": 40,
    "max_new_tokens": 512,
    "repetition_penalty": 1.1
}

# Generation
generation_output = model.generate(
    tokens,
    **generation_params
)

# Get the tokens from the output, decode them, print them
token_output = generation_output[0]
text_output = tokenizer.decode(token_output)
print(text_output)
```

**License**

Sailor is distributed under the terms of the Qwen License.