fastapi
uvicorn[standard]
transformers
torch 
torchaudio 
torchvision
pydantic
sentencepiece
accelerate>=0.26.0
gradio
git+https://github.com/abetlen/llama-cpp-python.git
bitsandbytes