tiiuae

/

Falcon3-1B-Base

like 23

Follow

Technology Innovation Institute 1.25k

Fixes: https://huggingface.co/tiiuae/Falcon3-1B-Base/discussions/13
There is no need to have a model.safetensors.index.json if the model weights are not sharded. It looks like this was a mistake at first place - e.g.: https://huggingface.co/tiiuae/Falcon3-1B-Instruct/tree/main does not have an index file

Tested locally if this PR does not break anything with the following snippet:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tiiuae/Falcon3-1B-Base"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto", revision="refs/pr/4")
tok = AutoTokenizer.from_pretrained(model_id)
print(model)

text = "The capital city of United States of America is"
inputs = tok(text, return_tensors="pt").to(0)
inputs.pop("token_type_ids", None)
out = model.generate(**inputs, max_new_tokens=10, do_sample=False)
print(tok.decode(out[0]))