FoundationVision
/

unitok_tokenizer

Feature Extraction

Model card Files Files and versions Community

unitok_tokenizer

3 contributors

History: 6 commits

machuofan's picture

Update README.md

cba5c1e verified 11 days ago

.gitattributes

1.52 kB

initial commit 16 days ago
README.md

3.62 kB

Update README.md 11 days ago
config.json

2.79 kB

update readme 12 days ago
unitok_tokenizer.pth
Detected Pickle imports (4)
- "torch.bfloat16",
- "torch.FloatStorage",
- "torch._utils._rebuild_tensor_v2",
- "collections.OrderedDict"
How to fix it?
3.31 GB
LFS

clean weights 16 days ago