Fixed req file v2

main/logs/llm_api.log

@@ -17,3 +17,15 @@
 2025-01-09 17:02:41,081 - llm_api - INFO - Loading model from source: huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
 2025-01-09 17:02:41,377 - llm_api - ERROR - Failed to initialize generation model huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install 'accelerate>=0.26.0'`
 2025-01-09 17:02:41,377 - api_routes - ERROR - Error initializing model: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install 'accelerate>=0.26.0'`
+2025-01-09 17:11:25,843 - hf_validation - WARNING - No .env file found. Fine if you're on Huggingface, but you need one to run locally on your PC.
+2025-01-09 17:11:25,843 - hf_validation - ERROR - No HF_TOKEN found in environment variables
+2025-01-09 17:11:25,843 - main - INFO - Starting LLM API server
+2025-01-09 17:11:25,843 - llm_api - INFO - Initializing LLM API
+2025-01-09 17:11:25,844 - llm_api - INFO - LLM API initialized successfully
+2025-01-09 17:11:25,844 - api_routes - INFO - Router initialized with LLM API instance
+2025-01-09 17:11:25,846 - main - INFO - FastAPI application created successfully
+2025-01-09 17:11:38,299 - api_routes - INFO - Received request to initialize model: huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
+2025-01-09 17:11:38,299 - llm_api - INFO - Initializing generation model: huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
+2025-01-09 17:11:38,299 - llm_api - INFO - Loading model from source: huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
+2025-01-09 17:11:38,487 - llm_api - ERROR - Failed to initialize generation model huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated: Using `bitsandbytes` 8-bit quantization requires the latest version of bitsandbytes: `pip install -U bitsandbytes`
+2025-01-09 17:11:38,487 - api_routes - ERROR - Error initializing model: Using `bitsandbytes` 8-bit quantization requires the latest version of bitsandbytes: `pip install -U bitsandbytes`

requirements.txt

@@ -1,6 +1,7 @@
 accelerate==1.2.1
 annotated-types==0.7.0
 anyio==4.8.0
+bitsandbytes==0.45.0
 certifi==2024.12.14
 charset-normalizer==3.4.1
 click==8.1.8