| # fastchat Nginx Gateway | |
| ## Purpose of the Gateway | |
| The Nginx gateway serves the following purposes: | |
| 1. Protects Gradio servers by acting as a firewall. | |
| 2. Facilitates dynamic mounting and unmounting of Gradio servers. | |
| 3. Provides load balancing for Gradio servers. | |
| 4. Offers additional security features, such as total connection limit. | |
| 5. Reduces attack surface by requiring only a single public port to be exposed for serving. | |
| ## Deployment and Updating of the Gateway | |
| ### Installing Nginx | |
| On Debian-based distributions (e.g., Ubuntu): | |
| ```bash | |
| sudo apt update | |
| sudo apt install nginx | |
| ``` | |
| On Red Hat-based distributions (e.g., CentOS, Fedora): | |
| ```bash | |
| sudo yum install epel-release | |
| sudo yum install nginx | |
| ``` | |
| ### Deployment | |
| Copy `nginx.conf` to `/etc/nginx/nginx.conf` (need sudo permission). | |
| Replace the port number 7860 in `server localhost:7860` with the port where you deploy the Gradio web server. | |
| Modify `upstream websocket` to configure Gradio servers behind the gateway. | |
| Lastly, update Nginx. | |
| ### HTTPS Deployment with a Public Domain URL | |
| Make sure you obtain the HTTPS certificate and the private key used to generate the certificate. | |
| Fill the addresses to your certificate and private key in the `[PATH_TO_SSL_CERT]` and `[PATH_TO_PRIVATE_KEY]` fields. | |
| If you have your own domain url to serve the chatbot, replace the chat.lmsys.org url with your own domain url. | |
| ### Updating | |
| Every time when `/etc/nginx/nginx.conf` is modified, you need to update the Nginx service: | |
| ```bash | |
| sudo nginx -t # check `/etc/nginx/nginx.conf` | |
| sudo systemctl reload nginx # restart Nginx service to load the new config | |
| sudo systemctl status nginx # check the status of the Nginx service. It should be active (running). | |
| ``` | |