chienweichang

/

Llama-3-Taiwan-8B-Instruct-DPO-GGUF

like 1

您好，我最近也在try利用llama.cpp去convert to GGUF
用的是70b的，但都會出現以下錯誤，想請問您是怎麼解決的，謝謝

root@olex8vnemo-69df4555c4-q6dlj://home/llama.cpp# python3 convert-hf-to-gguf.py /home/Llama-3-Taiwan-70B-Instruct --outtype f16 --outfile ggml-model-f16.gguf
INFO:hf-to-gguf:Loading model: Llama-3-Taiwan-70B-Instruct
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Set model parameters
INFO:hf-to-gguf:gguf: context length = 8192
INFO:hf-to-gguf:gguf: embedding length = 8192
INFO:hf-to-gguf:gguf: feed forward length = 28672
INFO:hf-to-gguf:gguf: head count = 64
INFO:hf-to-gguf:gguf: key-value head count = 8
INFO:hf-to-gguf:gguf: rope theta = 500000.0
INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05
INFO:hf-to-gguf:gguf: file type = 1
INFO:hf-to-gguf:Set model tokenizer
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
WARNING:hf-to-gguf:

WARNING:hf-to-gguf:**************************************************************************************
WARNING:hf-to-gguf:** WARNING: The BPE pre-tokenizer was not recognized!
WARNING:hf-to-gguf:**          There are 2 possible reasons for this:
WARNING:hf-to-gguf:**          - the model has not been added to convert-hf-to-gguf-update.py yet
WARNING:hf-to-gguf:**          - the pre-tokenization config has changed upstream
WARNING:hf-to-gguf:**          Check your model files and convert-hf-to-gguf-update.py and update them accordingly.
WARNING:hf-to-gguf:** ref:     https://github.com/ggerganov/llama.cpp/pull/6920
WARNING:hf-to-gguf:**
WARNING:hf-to-gguf:** chkhsh:  0ef9807a4087ebef797fc749390439009c3b9eda9ad1a097abbe738f486c01e5
WARNING:hf-to-gguf:**************************************************************************************
WARNING:hf-to-gguf:

Traceback (most recent call last):
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 1306, in set_vocab
    self. _set_vocab_sentencepiece()
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 574, in _set_vocab_sentencepiece
    tokens, scores, toktypes = self._create_vocab_sentencepiece()
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 595, in _create_vocab_sentencepiece
    raise FileNotFoundError(f"File not found: {tokenizer_path}")
FileNotFoundError: File not found: /home/Llama-3-Taiwan-70B-Instruct/tokenizer.model

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 1309, in set_vocab
    self._set_vocab_llama_hf()
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 650, in _set_vocab_llama_hf
    vocab = gguf.LlamaHfVocab(self.dir_model)
  File "/home/llama.cpp/gguf-py/gguf/vocab.py", line 362, in __init__
    raise TypeError('Llama 3 must be converted with BpeVocab')
TypeError: Llama 3 must be converted with BpeVocab

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 3254, in <module>
    main()
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 3238, in main
    model_instance.set_vocab()
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 1312, in set_vocab
    self._set_vocab_gpt2()
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 510, in _set_vocab_gpt2
    tokens, toktypes, tokpre = self.get_vocab_base()
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 386, in get_vocab_base
    tokpre = self.get_vocab_base_pre(tokenizer)
  File "/home/llama.cpp/convert-hf-to-gguf.py", line 501, in get_vocab_base_pre
    raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base