Nous Hermes 2 - Mistral 7B - DPO - AWQ

image/png

Model Description

This repo contains the AWQ quantized version of the Nous Hermes 2 - Mistral 7B - DPO model. It was quantized with AutoAWQ using the following settings:

{"zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM"}
Downloads last month
27
Safetensors
Model size
1.2B params
Tensor type
I32
·
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for heyholetsgo/Nous-Hermes-2-Mistral-7B-DPO-AWQ

Quantized
(9)
this model

Dataset used to train heyholetsgo/Nous-Hermes-2-Mistral-7B-DPO-AWQ