INT8 ONNX version of cmarkea/bloomz-560m-sft-chat to use with Transformers.js.

Example usage

Pipeline API

import { pipeline } from '@xenova/transformers';

const generator = await pipeline('text-generation', 'Felladrin/onnx-bloomz-560m-sft-chat');
const output =  await generator('</s>What is the first world war?<s>', { add_special_tokens: true, max_new_tokens: 128, repetition_penalty: 1.2});
console.log(output); // The first world war was a conflict between the United States and the Soviet Union. The conflict began in World War II and lasted until the end of World War III.

Auto Classes

import { AutoModelForCausalLM, AutoTokenizer } from '@xenova/transformers';

const model_path = 'Felladrin/onnx-bloomz-560m-sft-chat';
const model = await AutoModelForCausalLM.from_pretrained(model_path);
const tokenizer = await AutoTokenizer.from_pretrained(model_path);

const prompt = '</s>What is the first world war?<s>';
const { input_ids } = tokenizer(prompt);
const tokens = await model.generate(input_ids, { max_new_tokens: 128, repetition_penalty: 1.2});
console.log(tokenizer.decode(tokens[0], { skip_special_tokens: true }));
// The first world war was a conflict between the United States and the Soviet Union. The conflict began in World War II and lasted until the end of World War III.
Downloads last month
15
Inference Examples
Inference API (serverless) does not yet support transformers.js models for this pipeline type.

Model tree for Felladrin/onnx-bloomz-560m-sft-chat

Quantized
(1)
this model

Datasets used to train Felladrin/onnx-bloomz-560m-sft-chat