|
--- |
|
title: Turing-Test-Prompt-Competition |
|
app_file: eval.py |
|
sdk: gradio |
|
sdk_version: 4.44.0 |
|
--- |
|
# Turing-Test-Prompt-Competition |
|
|
|
This project implements a chatbot using vLLM for inference and Streamlit for the user interface and Gradio for the evaluation interface. |
|
|
|
## Setup and Deployment |
|
|
|
### vLLM Deployment |
|
|
|
To deploy vLLM: |
|
|
|
1. Download the LLaMA model: |
|
``` |
|
modal run download_llama.py |
|
``` |
|
|
|
2. Deploy the vLLM inference service: |
|
``` |
|
modal deploy vllm_inference.py |
|
``` |
|
|
|
### Running the Chatbot |
|
|
|
To run the chatbot locally: |
|
|
|
1. Start the Streamlit app: |
|
``` |
|
streamlit run chatbot.py |
|
``` |
|
|
|
2. To make the chatbot accessible over the internet, use ngrok: |
|
``` |
|
ngrok http 8501 |
|
``` |
|
|
|
### Running the Evaluation Interface |
|
|
|
To run the evaluation interface locally: |
|
|
|
1. Start the Gradio app: |
|
``` |
|
gradio eval.py |
|
``` |
|
|
|
2. To deploy to HF Space, run: |
|
``` |
|
gradio deploy |
|
``` |
|
|
|
## Project Structure |
|
|
|
- `download_llama.py`: Script to download the LLaMA model |
|
- `vllm_inference.py`: vLLM inference service |
|
- `chatbot.py`: Streamlit-based chatbot interface |
|
|
|
## License |
|
|
|
This project is licensed under the GNU Affero General Public License v3.0. See the [LICENSE](LICENSE) file for details. |
|
|
|
## Contributing |
|
|
|
Contributions are welcome! Please feel free to submit a Pull Request. |
|
|
|
## Support |
|
|
|
If you encounter any problems or have any questions, please open an issue in this repository. |
