Spaces:
Runtime error
Runtime error
File size: 31,041 Bytes
14a71e2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 |
{
"cells": [
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting unsloth@ git+https://github.com/unslothai/unsloth.git (from unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git)\n",
" Cloning https://github.com/unslothai/unsloth.git to /tmp/pip-install-iyfa2a6n/unsloth_fb5a1c9f71c749ed87cb3c95c10067c7\n",
" Running command git clone --filter=blob:none --quiet https://github.com/unslothai/unsloth.git /tmp/pip-install-iyfa2a6n/unsloth_fb5a1c9f71c749ed87cb3c95c10067c7\n",
" Resolved https://github.com/unslothai/unsloth.git to commit 976d11a10d54383aeb7a692c69e01151a20bfd72\n",
" Installing build dependencies ... \u001b[?25ldone\n",
"\u001b[?25h Getting requirements to build wheel ... \u001b[?25ldone\n",
"\u001b[?25h Preparing metadata (pyproject.toml) ... \u001b[?25ldone\n",
"\u001b[?25hRequirement already satisfied: packaging in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (24.1)\n",
"Requirement already satisfied: tyro in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.8.10)\n",
"Requirement already satisfied: transformers>=4.43.2 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (4.43.3)\n",
"Requirement already satisfied: datasets>=2.16.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2.20.0)\n",
"Requirement already satisfied: sentencepiece>=0.2.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.2.0)\n",
"Requirement already satisfied: tqdm in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (4.66.4)\n",
"Requirement already satisfied: psutil in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (6.0.0)\n",
"Requirement already satisfied: wheel>=0.42.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.43.0)\n",
"Requirement already satisfied: numpy in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (1.23.5)\n",
"Requirement already satisfied: protobuf<4.0.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (3.20.3)\n",
"Requirement already satisfied: huggingface-hub in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.24.3)\n",
"Requirement already satisfied: hf-transfer in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.1.8)\n",
"Requirement already satisfied: filelock in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (3.15.4)\n",
"Requirement already satisfied: pyarrow>=15.0.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (17.0.0)\n",
"Requirement already satisfied: pyarrow-hotfix in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.6)\n",
"Requirement already satisfied: dill<0.3.9,>=0.3.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.3.8)\n",
"Requirement already satisfied: pandas in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2.2.2)\n",
"Requirement already satisfied: requests>=2.32.2 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2.32.3)\n",
"Requirement already satisfied: xxhash in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (3.4.1)\n",
"Requirement already satisfied: multiprocess in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.70.16)\n",
"Requirement already satisfied: fsspec<=2024.5.0,>=2023.1.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from fsspec[http]<=2024.5.0,>=2023.1.0->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2024.5.0)\n",
"Requirement already satisfied: aiohttp in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (3.9.5)\n",
"Requirement already satisfied: pyyaml>=5.1 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (6.0.1)\n",
"Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from huggingface-hub->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (4.12.2)\n",
"Requirement already satisfied: regex!=2019.12.17 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from transformers>=4.43.2->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2024.7.24)\n",
"Requirement already satisfied: safetensors>=0.4.1 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from transformers>=4.43.2->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.4.3)\n",
"Requirement already satisfied: tokenizers<0.20,>=0.19 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from transformers>=4.43.2->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.19.1)\n",
"Requirement already satisfied: docstring-parser>=0.16 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from tyro->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.16)\n",
"Requirement already satisfied: rich>=11.1.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from tyro->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (13.7.1)\n",
"Requirement already satisfied: shtab>=1.5.6 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from tyro->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (1.7.1)\n",
"Requirement already satisfied: eval-type-backport>=0.1.3 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from tyro->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.2.0)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from aiohttp->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (1.2.0)\n",
"Requirement already satisfied: attrs>=17.3.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from aiohttp->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (23.1.0)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from aiohttp->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (1.4.0)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from aiohttp->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (6.0.4)\n",
"Requirement already satisfied: yarl<2.0,>=1.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from aiohttp->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (1.9.3)\n",
"Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from aiohttp->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (4.0.3)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from requests>=2.32.2->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (3.3.2)\n",
"Requirement already satisfied: idna<4,>=2.5 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from requests>=2.32.2->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (3.7)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from requests>=2.32.2->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2.2.2)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from requests>=2.32.2->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2024.7.4)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from rich>=11.1.0->tyro->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from rich>=11.1.0->tyro->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2.18.0)\n",
"Requirement already satisfied: python-dateutil>=2.8.2 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from pandas->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2.9.0)\n",
"Requirement already satisfied: pytz>=2020.1 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from pandas->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2024.1)\n",
"Requirement already satisfied: tzdata>=2022.7 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from pandas->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (2024.1)\n",
"Requirement already satisfied: mdurl~=0.1 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from markdown-it-py>=2.2.0->rich>=11.1.0->tyro->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (0.1.2)\n",
"Requirement already satisfied: six>=1.5 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->datasets>=2.16.0->unsloth@ git+https://github.com/unslothai/unsloth.git->unsloth[colab-new]@ git+https://github.com/unslothai/unsloth.git) (1.16.0)\n",
"Requirement already satisfied: xformers<0.0.27 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (0.0.26.post1)\n",
"Requirement already satisfied: trl<0.9.0 in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (0.8.6)\n",
"Requirement already satisfied: peft in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (0.12.0)\n",
"Requirement already satisfied: accelerate in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (0.33.0)\n",
"Requirement already satisfied: bitsandbytes in /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages (0.43.3)\n"
]
}
],
"source": [
"!pip install \"unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git\"\n",
"!pip install --no-deps \"xformers<0.0.27\" \"trl<0.9.0\" peft accelerate bitsandbytes"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-09-02 22:16:32.351723: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
"2024-09-02 22:16:32.441229: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
"2024-09-02 22:16:32.454471: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
"2024-09-02 22:16:32.544617: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
"To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
"2024-09-02 22:16:33.579642: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
"WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:\n",
" PyTorch 2.3.0+cu121 with CUDA 1201 (you have 2.4.0+cu121)\n",
" Python 3.9.19 (you have 3.9.19)\n",
" Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)\n",
" Memory-efficient attention, SwiGLU, sparse and more won't be available.\n",
" Set XFORMERS_MORE_DETAILS=1 for more details\n"
]
},
{
"ename": "ImportError",
"evalue": "Unsloth: Xformers was not installed correctly.\nPlease install xformers separately first.\nThen confirm if it's correctly installed by running:\npython -m xformers.info\n\nLonger error message:\nxFormers can't load C++/CUDA extensions. xFormers was built for:\n PyTorch 2.3.0+cu121 with CUDA 1201 (you have 2.4.0+cu121)\n Python 3.9.19 (you have 3.9.19)\n Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)\n Memory-efficient attention, SwiGLU, sparse and more won't be available.",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mOSError\u001b[0m Traceback (most recent call last)",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/xformers/_cpp_lib.py:128\u001b[0m, in \u001b[0;36m_register_extensions\u001b[0;34m()\u001b[0m\n\u001b[1;32m 127\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 128\u001b[0m \u001b[43mtorch\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mops\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mload_library\u001b[49m\u001b[43m(\u001b[49m\u001b[43mext_specs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43morigin\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 129\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mOSError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m exc:\n",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/torch/_ops.py:1295\u001b[0m, in \u001b[0;36m_Ops.load_library\u001b[0;34m(self, path)\u001b[0m\n\u001b[1;32m 1291\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m dl_open_guard():\n\u001b[1;32m 1292\u001b[0m \u001b[38;5;66;03m# Import the shared library into the process, thus running its\u001b[39;00m\n\u001b[1;32m 1293\u001b[0m \u001b[38;5;66;03m# static (global) initialization code in order to register custom\u001b[39;00m\n\u001b[1;32m 1294\u001b[0m \u001b[38;5;66;03m# operators with the JIT.\u001b[39;00m\n\u001b[0;32m-> 1295\u001b[0m \u001b[43mctypes\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCDLL\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpath\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1296\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mloaded_libraries\u001b[38;5;241m.\u001b[39madd(path)\n",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/ctypes/__init__.py:382\u001b[0m, in \u001b[0;36mCDLL.__init__\u001b[0;34m(self, name, mode, handle, use_errno, use_last_error, winmode)\u001b[0m\n\u001b[1;32m 381\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m handle \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m--> 382\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_handle \u001b[38;5;241m=\u001b[39m \u001b[43m_dlopen\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmode\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 383\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n",
"\u001b[0;31mOSError\u001b[0m: /home/nihaltm/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/xformers/_C.so: undefined symbol: _ZN3c1010Dispatcher17runRecordFunctionERN2at14RecordFunctionESt17reference_wrapperIKNS_14FunctionSchemaEENS_11DispatchKeyE",
"\nThe above exception was the direct cause of the following exception:\n",
"\u001b[0;31mxFormersInvalidLibException\u001b[0m Traceback (most recent call last)",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/unsloth/models/_utils.py:253\u001b[0m\n\u001b[1;32m 252\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 253\u001b[0m \u001b[43m_register_extensions\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;66;03m# Check if C++ modules are loaded correctly\u001b[39;00m\n\u001b[1;32m 254\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m error:\n",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/xformers/_cpp_lib.py:130\u001b[0m, in \u001b[0;36m_register_extensions\u001b[0;34m()\u001b[0m\n\u001b[1;32m 129\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mOSError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m exc:\n\u001b[0;32m--> 130\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m xFormersInvalidLibException(build_metadata) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mexc\u001b[39;00m\n\u001b[1;32m 131\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m build_metadata\n",
"\u001b[0;31mxFormersInvalidLibException\u001b[0m: xFormers can't load C++/CUDA extensions. xFormers was built for:\n PyTorch 2.3.0+cu121 with CUDA 1201 (you have 2.4.0+cu121)\n Python 3.9.19 (you have 3.9.19)\n Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)\n Memory-efficient attention, SwiGLU, sparse and more won't be available.",
"\nDuring handling of the above exception, another exception occurred:\n",
"\u001b[0;31mImportError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[2], line 5\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mdatasets\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m load_dataset\n\u001b[1;32m 4\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mtransformers\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m TrainingArguments, TextStreamer\n\u001b[0;32m----> 5\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01munsloth\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchat_templates\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m get_chat_template\n\u001b[1;32m 6\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01munsloth\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m FastLanguageModel, is_bfloat16_supported\n",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/unsloth/__init__.py:154\u001b[0m\n\u001b[1;32m 144\u001b[0m warnings\u001b[38;5;241m.\u001b[39mwarn(\n\u001b[1;32m 145\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mUnsloth: CUDA is not linked properly.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\\\n\u001b[1;32m 146\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mTry running `python -m bitsandbytes` then `python -m xformers.info`\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\\\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 150\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mUnsloth will still run for now, but maybe it might crash - let\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124ms hope it works!\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 151\u001b[0m )\n\u001b[1;32m 152\u001b[0m \u001b[38;5;28;01mpass\u001b[39;00m\n\u001b[0;32m--> 154\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mmodels\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;241m*\u001b[39m\n\u001b[1;32m 155\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01msave\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;241m*\u001b[39m\n\u001b[1;32m 156\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchat_templates\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;241m*\u001b[39m\n",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/unsloth/models/__init__.py:15\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;66;03m# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.\u001b[39;00m\n\u001b[1;32m 2\u001b[0m \u001b[38;5;66;03m#\u001b[39;00m\n\u001b[1;32m 3\u001b[0m \u001b[38;5;66;03m# Licensed under the Apache License, Version 2.0 (the \"License\");\u001b[39;00m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[38;5;66;03m# See the License for the specific language governing permissions and\u001b[39;00m\n\u001b[1;32m 13\u001b[0m \u001b[38;5;66;03m# limitations under the License.\u001b[39;00m\n\u001b[0;32m---> 15\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mloader\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m FastLanguageModel\n\u001b[1;32m 16\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mllama\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m FastLlamaModel\n\u001b[1;32m 17\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mmistral\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m FastMistralModel\n",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/unsloth/models/loader.py:15\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;66;03m# Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.\u001b[39;00m\n\u001b[1;32m 2\u001b[0m \u001b[38;5;66;03m#\u001b[39;00m\n\u001b[1;32m 3\u001b[0m \u001b[38;5;66;03m# Licensed under the Apache License, Version 2.0 (the \"License\");\u001b[39;00m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[38;5;66;03m# See the License for the specific language governing permissions and\u001b[39;00m\n\u001b[1;32m 13\u001b[0m \u001b[38;5;66;03m# limitations under the License.\u001b[39;00m\n\u001b[0;32m---> 15\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01m_utils\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m is_bfloat16_supported, HAS_FLASH_ATTENTION, HAS_FLASH_ATTENTION_SOFTCAPPING\n\u001b[1;32m 16\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mllama\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m FastLlamaModel, logger\n\u001b[1;32m 17\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mmistral\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m FastMistralModel\n",
"File \u001b[0;32m~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/unsloth/models/_utils.py:255\u001b[0m\n\u001b[1;32m 253\u001b[0m _register_extensions() \u001b[38;5;66;03m# Check if C++ modules are loaded correctly\u001b[39;00m\n\u001b[1;32m 254\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m error:\n\u001b[0;32m--> 255\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mImportError\u001b[39;00m(\n\u001b[1;32m 256\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mUnsloth: Xformers was not installed correctly.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\\\n\u001b[1;32m 257\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mPlease install xformers separately first.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\\\n\u001b[1;32m 258\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mThen confirm if it\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124ms correctly installed by running:\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\\\n\u001b[1;32m 259\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpython -m xformers.info\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 260\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mLonger error message:\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;241m+\u001b[39m \u001b[38;5;28mstr\u001b[39m(error)\n\u001b[1;32m 261\u001b[0m )\n\u001b[1;32m 262\u001b[0m \u001b[38;5;28;01mpass\u001b[39;00m\n\u001b[1;32m 263\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mxformers\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mops\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mfmha\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mxformers\u001b[39;00m\n",
"\u001b[0;31mImportError\u001b[0m: Unsloth: Xformers was not installed correctly.\nPlease install xformers separately first.\nThen confirm if it's correctly installed by running:\npython -m xformers.info\n\nLonger error message:\nxFormers can't load C++/CUDA extensions. xFormers was built for:\n PyTorch 2.3.0+cu121 with CUDA 1201 (you have 2.4.0+cu121)\n Python 3.9.19 (you have 3.9.19)\n Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)\n Memory-efficient attention, SwiGLU, sparse and more won't be available."
]
}
],
"source": [
"import torch\n",
"from trl import SFTTrainer\n",
"from datasets import load_dataset\n",
"from transformers import TrainingArguments, TextStreamer\n",
"from unsloth.chat_templates import get_chat_template\n",
"from unsloth import FastLanguageModel, is_bfloat16_supported"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "tf-gpu",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.19"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
|