Spaces:
Running
on
T4
Running
on
T4
Scaling App
#33
by
aneerimmco
- opened
@annasun28 Is the app scalable by default? please let me know your thoughts on using with Ray.serve or NVIDIA Triton or Kubernetes?
It's not designed for many users to be streaming input audio to the same server, unfortunately. In HF spaces we scale it by setting up replicas so that different users are connected to different servers. We'd welcome contributions to make the service more scalable!