Usage

import torch
from q8_kernels.models.T5EncoderFP8 import T5EncoderModelFP8
text_encoder = T5EncoderModelFP8.from_pretrained(
    "konakona/t5xxl_encoder_fp8", torch_dtype=torch.bfloat16
)

Needed dX for token training(PTI and textual inversion) in LTX.
dX is in bf16, calculations in FP8
Needs Ada GPU

Downloads last month: 23

Safetensors

Model size

4.76B params

Tensor type

F32

BF16

F8_E4M3

Inference Providers NEW

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.