Tokenizer not found error
2023-07-26 14:57:46.040538: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-07-26 14:57:46.139528: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0.
2023-07-26 14:57:46.161197: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-07-26 14:57:46.533739: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-07-26 14:57:46.533779: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-07-26 14:57:46.533783: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Downloading (…)lve/main/config.json: 100%|██████████████████████████████████████████████| 1.00k/1.00k [00:00<00:00, 184kB/s]
Downloading (…)"pytorch_model.bin";: 100%|███████████████████████████████████████████████| 510M/510M [00:23<00:00, 21.3MB/s]
Traceback (most recent call last):
File "test.py", line 23, in
pipe = pipeline("text-classification", model="curt-tigges/gpt2-imdb-sentiment-classifier")
File "/xx/xxx/miniconda3/envs/huggfacegpu/lib/python3.7/site-packages/transformers/pipelines/init.py", line 874, in pipeline
tokenizer_identifier, use_fast=use_fast, _from_pipeline=task, **hub_kwargs, **tokenizer_kwargs
File "/xx/xxx/miniconda3/envs/huggfacegpu/lib/python3.7/site-packages/transformers/models/auto/tokenization_auto.py", line 698, in from_pretrained
return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/xx/xxx/miniconda3/envs/huggfacegpu/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1790, in from_pretrained
f"Can't load tokenizer for '{pretrained_model_name_or_path}'. If you were trying to load it from "
OSError: Can't load tokenizer for 'curt-tigges/gpt2-imdb-sentiment-classifier'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'curt-tigges/gpt2-imdb-sentiment-classifier' is the correct path to a directory containing all relevant files for a GPT2TokenizerFast tokenizer.
Any ideas why?