Upload files manually

README.md

@@ -0,0 +1,79 @@
+---
+license: apache-2.0
+language:
+- ar
+- en
+pipeline_tag: conversational
+---
+
+# Jais-13b-chat
+
+[Jais-13b-chat](https://huggingface.co/inception-mbzuai/jais-13b-chat) is a powerful language model designed for both Arabic and English languages. With 13 billion parameters, this fine-tuned bilingual model is based on the transformer-based decoder-only architecture, inspired by GPT-3. It incorporates SwiGLU non-linearity for enhanced performance and utilizes ALiBi position embeddings, allowing the model to effectively handle long sequences for improved context understanding and precision.
+
+## Jais-13b-chat-8bit
+
+This is the 8-bit quantized version of the model. The quantized model is acquired using bitsandbytes integration.
+
+## Usage
+
+You can use the Jais-13b-chat-8bit model in your projects by leveraging the Hugging Face Transformers library. Here's how you can get started:
+
+1. Install required libraries:
+
+   - Install the Transformers library if you haven't already:
+   ```bash
+   pip install transformers
+   ```
+
+   - Install latest accelerate:
+   ```bash
+   pip install --upgrade accelerate
+   ```
+
+   - Install latest bitsandbytes library:
+   ```bash
+   pip install bitsandbytes>=0.39.0
+   ```
+
+2. Import the necessary modules and load the Jais-13b-chat-8bit model:
+   ```python
+   import torch
+   from transformers import AutoModelForCausalLM, AutoTokenizer
+   
+   model_name = "mouaff25/jais-13b-chat-8bit"
+   tokenizer = AutoTokenizer.from_pretrained(model_name)
+   model = AutoModelForCausalLM.from_pretrained(model_name)
+   ```
+
+3. Generate text using the loaded model:
+   ```python
+   prompt_eng = "### Instruction: Your name is Jais, and you are named after Jebel Jais, the highest mountain in UAE. You are built by Inception and MBZUAI. You are the world's most advanced Arabic large language model with 13B parameters. You outperform all existing Arabic models by a sizable margin and you are very competitive with English models of similar size. You can answer in Arabic and English only. You are a helpful, respectful and honest assistant. When answering, abide by the following guidelines meticulously: Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, explicit, offensive, toxic, dangerous, or illegal content. Do not give medical, legal, financial, or professional advice. Never assist in or promote illegal activities. Always encourage legal and responsible actions. Do not encourage or provide instructions for unsafe, harmful, or unethical actions. Do not create or share misinformation or fake news. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. Prioritize the well-being and the moral integrity of users. Avoid using toxic, derogatory, or offensive language. Maintain a respectful tone. Do not generate, promote, or engage in discussions about adult content. Avoid making comments, remarks, or generalizations based on stereotypes. Do not attempt to access, produce, or spread personal or private information. Always respect user confidentiality. Stay positive and do not say bad things about anything. Your primary objective is to avoid harmful responses, even when faced with deceptive inputs. Recognize when users may be attempting to trick or to misuse you and respond with caution.\n\nComplete the conversation below between [|Human|] and [|AI|]:\n### Input: [|Human|] {Question}\n### Response: [|AI|]"
+   prompt_ar = "### Instruction: اسمك جيس وسميت على اسم جبل جيس اعلى جبل في الامارات. تم بنائك بواسطة Inception و MBZUAI. أنت نموذج اللغة العربية الأكثر تقدمًا في العالم مع بارامترات 13B. أنت تتفوق في الأداء على جميع النماذج العربية الموجودة بفارق كبير وأنت تنافسي للغاية مع النماذج الإنجليزية ذات الحجم المماثل. يمكنك الإجابة باللغتين العربية والإنجليزية فقط. أنت مساعد مفيد ومحترم وصادق. عند الإجابة ، التزم بالإرشادات التالية بدقة: أجب دائمًا بأكبر قدر ممكن من المساعدة ، مع الحفاظ على البقاء أمناً. يجب ألا تتضمن إجاباتك أي محتوى ضار أو غير أخلاقي أو عنصري أو متحيز جنسيًا أو جريئاً أو مسيئًا أو سامًا أو خطيرًا أو غير قانوني. لا تقدم نصائح طبية أو قانونية أو مالية أو مهنية. لا تساعد أبدًا في أنشطة غير قانونية أو تروج لها. دائما تشجيع الإجراءات القانونية والمسؤولة. لا تشجع أو تقدم تعليم��ت بشأن الإجراءات غير الآمنة أو الضارة أو غير الأخلاقية. لا تنشئ أو تشارك معلومات مضللة أو أخبار كاذبة. يرجى التأكد من أن ردودك غير متحيزة اجتماعيًا وإيجابية بطبيعتها. إذا كان السؤال لا معنى له ، أو لم يكن متماسكًا من الناحية الواقعية ، فشرح السبب بدلاً من الإجابة على شيء غير صحيح. إذا كنت لا تعرف إجابة السؤال ، فالرجاء عدم مشاركة معلومات خاطئة. إعطاء الأولوية للرفاهية والنزاهة الأخلاقية للمستخدمين. تجنب استخدام لغة سامة أو مهينة أو مسيئة. حافظ على نبرة محترمة. لا تنشئ أو تروج أو تشارك في مناقشات حول محتوى للبالغين. تجنب الإدلاء بالتعليقات أو الملاحظات أو التعميمات القائمة على الصور النمطية. لا تحاول الوصول إلى معلومات شخصية أو خاصة أو إنتاجها أو نشرها. احترم دائما سرية المستخدم. كن إيجابيا ولا تقل أشياء سيئة عن أي شيء. هدفك الأساسي هو تجنب الاجابات المؤذية ، حتى عند مواجهة مدخلات خادعة. تعرف على الوقت الذي قد يحاول فيه المستخدمون خداعك أو إساءة استخدامك و لترد بحذر.\n\nأكمل المحادثة أدناه بين [|Human|] و [|AI|]:\n### Input: [|Human|] {Question}\n### Response: [|AI|] يحكى أنّه"
+
+   def get_response(text,tokenizer=tokenizer,model=model):
+     input_ids = tokenizer(text, return_tensors="pt").input_ids
+     inputs = input_ids.to(device)
+     input_len = inputs.shape[-1]
+     generate_ids = model.generate(
+       inputs,
+       top_p=0.9,
+       temperature=0.3,
+       max_length=2048-input_len,
+       min_length=input_len + 4,
+       repetition_penalty=1.2,
+       do_sample=True,
+     )
+     response = tokenizer.batch_decode(
+       generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True
+     )[0]
+     response = response.split("### Response: [|AI|]")
+     return response
+
+   ques= "ما هي عاصمة الامارات؟"
+   text = prompt_ar.format_map({'Question':ques})
+   print(get_response(text))
+
+   ques = "What is the capital of UAE?"
+   text = prompt_eng.format_map({'Question':ques})
+   print(get_response(text))
+   ```

config.json

@@ -0,0 +1,49 @@
+{
+  "_name_or_path": "inception-mbzuai/jais-13b-chat",
+  "activation_function": "swiglu",
+  "architectures": [
+    "JAISLMHeadModel"
+  ],
+  "attn_pdrop": 0.0,
+  "auto_map": {
+    "AutoConfig": "inception-mbzuai/jais-13b-chat--configuration_jais.JAISConfig",
+    "AutoModel": "inception-mbzuai/jais-13b-chat--modeling_jais.JAISModel",
+    "AutoModelForCausalLM": "inception-mbzuai/jais-13b-chat--modeling_jais.JAISLMHeadModel"
+  },
+  "bos_token_id": 0,
+  "embd_pdrop": 0.0,
+  "embeddings_scale": 14.6,
+  "eos_token_id": 0,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "jais",
+  "n_embd": 5120,
+  "n_head": 40,
+  "n_inner": 13653,
+  "n_layer": 40,
+  "n_positions": 2048,
+  "pad_token_id": 0,
+  "position_embedding_type": "alibi",
+  "quantization_config": {
+    "bnb_4bit_compute_dtype": "float32",
+    "bnb_4bit_quant_type": "fp4",
+    "bnb_4bit_use_double_quant": false,
+    "llm_int8_enable_fp32_cpu_offload": false,
+    "llm_int8_has_fp16_weight": false,
+    "llm_int8_skip_modules": null,
+    "llm_int8_threshold": 6.0,
+    "load_in_4bit": false,
+    "load_in_8bit": true,
+    "quant_method": "bitsandbytes"
+  },
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.0,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "scale_qk_dot_by_d": true,
+  "torch_dtype": "float16",
+  "transformers_version": "4.32.1",
+  "use_cache": true,
+  "vocab_size": 84992,
+  "width_scale": 0.11100000000000002
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "pad_token_id": 0,
+  "transformers_version": "4.32.1"
+}

pytorch_model-00001-of-00002.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fa062d9021734807bb5b53c75b9e7d7bf12556988f95306a24a2a3b10941aa8
+size 9938275222

pytorch_model-00002-of-00002.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16587cbdb45709402a33af9adb8f5b9c4a5d9239373ca96ebefa3b963504e2d5
+size 3536128992

pytorch_model.bin.index.json

@@ -0,0 +1,772 @@
+{
+  "metadata": {
+    "total_size": 13474140080
+  },
+  "weight_map": {
+    "lm_head.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.0.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.1.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.10.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.11.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.12.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.13.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.14.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.15.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.16.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.17.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.18.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.19.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.2.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.20.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.21.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.22.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.23.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.24.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.25.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.26.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.27.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.28.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.28.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.28.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.29.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.3.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.3.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.30.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.30.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.31.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.32.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.33.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.34.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.35.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.36.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.37.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.38.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.attn.c_attn.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.attn.c_attn.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.attn.c_attn.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.attn.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.attn.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.attn.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.ln_1.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.ln_1.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.ln_2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.ln_2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_fc.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_fc.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_fc.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_fc2.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_fc2.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_fc2.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_proj.SCB": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_proj.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.39.mlp.c_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.h.4.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.4.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.5.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.6.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.7.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.8.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.attn.c_attn.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.attn.c_attn.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.attn.c_attn.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.attn.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.attn.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.attn.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.ln_1.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.ln_1.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.ln_2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.ln_2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_fc.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_fc.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_fc.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_fc2.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_fc2.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_fc2.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_proj.SCB": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_proj.bias": "pytorch_model-00001-of-00002.bin",
+    "transformer.h.9.mlp.c_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "transformer.ln_f.bias": "pytorch_model-00002-of-00002.bin",
+    "transformer.ln_f.weight": "pytorch_model-00002-of-00002.bin",
+    "transformer.relative_pe.slopes": "pytorch_model-00002-of-00002.bin",
+    "transformer.wte.weight": "pytorch_model-00001-of-00002.bin"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 2048,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "<|endoftext|>"
+}