how to get inference?
#1
by
amriteshbytesview
- opened
after loading the model how to get inference from model for NER
Use unsloth to get inference.
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "Debk/Ben_NER_model", # YOUR MODEL YOU USED FOR TRAINING
max_seq_length = max_seq_length,
dtype = dtype,
load_in_4bit = load_in_4bit,
)
FastLanguageModel.for_inference(model)