File size: 4,566 Bytes
055fea6
 
 
 
 
 
 
 
 
d4afeb7
 
bfb73a4
32985b7
cbbce32
 
bfb73a4
06d3ffa
897989b
 
bfb73a4
06d3ffa
b62fb09
 
 
20b1101
 
9dfde9a
d4afeb7
 
 
 
 
 
 
 
 
9aa91fc
fdec844
20b1101
 
d4afeb7
 
 
 
0613780
d4afeb7
 
3b19079
d4afeb7
fdec844
 
0b53a03
0613780
fdec844
d4afeb7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b7ad1d4
fdec844
 
a56d2da
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b62fb09
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
---
title: README
emoji: 🌍
colorFrom: indigo
colorTo: indigo
sdk: static
pinned: false
---

# ZeroGPU Spaces

<div style="background-color: rgba(255, 255, 255, 0.5); color: rgba(0, 0, 0, 0.87); border: 1px solid rgba(0, 0, 0, 0.5); border-radius: 8px; padding: 0.5rem 1rem;">
  <span style="font-weight: 600;">ZeroGPU is currently in beta.</span> It's available to use for everyone and <a target="_blank" href="https://huggingface.co/subscribe/pro" style="color: inherit;">PRO</a> users can host their own ZeroGPU Spaces.
</div>

<div style="background-color: rgba(255, 255, 255, 0.5); color: rgba(0, 0, 0, 0.87); border: 1px solid rgba(0, 0, 0, 0.5); border-radius: 8px; padding: 0.5rem 1rem; margin-top: 10px;">
  <span style="font-weight: 600;">Please share your feedback about ZeroGPU</span> in the <a target="_blank" href="https://huggingface.co/spaces/zero-gpu-explorers/README/discussions" style="color: inherit;">Community tab</a>.
</div>

<div style="background-color: rgba(255, 255, 255, 0.5); color: rgba(0, 0, 0, 0.87); border: 1px solid rgba(0, 0, 0, 0.5); border-radius: 8px; padding: 0.5rem 1rem; margin-top: 10px;">
  <span style="font-weight: 600;">Browse ZeroGPU Spaces</span> here: <a target="_blank" href="https://huggingface.co/spaces/enzostvs/zero-gpu-spaces" style="color: inherit;">Dedicated list</a>.
</div>


<img src="https://cdn-uploads.huggingface.co/production/uploads/5f17f0a0925b9863e28ad517/cAlvAOu9QC547zrmRVpS5.gif" style="width:100%;"/>


*ZeroGPU* is a new kind of hardware for Spaces.

It has two goals :
- Provide **free GPU access** for Spaces
- Allow Spaces to run on **multiple GPUs**

This is achieved by making Spaces efficiently hold and release GPUs as needed
(as opposed to a classical GPU Space that holds exactly one GPU at any point in time)

ZeroGPU uses _Nvidia A100_ GPU devices under the hood (40GB of vRAM are available for each workloads)

<img src="https://cdn-uploads.huggingface.co/production/uploads/5f17f0a0925b9863e28ad517/naVZI-v41zNxmGlhEhGDJ.gif" style="width: 100%; max-width:550px"/>


# Compatibility

*ZeroGPU* Spaces should mostly be compatible with any PyTorch-based GPU Space.<br>
Compatibility with high level HF libraries like `transformers` or `diffusers` is slightly more guaranteed<br>
That said, ZeroGPU Spaces are not as broadly compatible as classical GPU Spaces and you might still encounter unexpected bugs

Also, for now, ZeroGPU Spaces only works with the **Gradio SDK**

Supported versions:
- Gradio: 4+
- PyTorch: All versions from `2.0.0` to `2.2.0`
- Python: `3.10.13`

# Usage

In order to make your Space work with ZeroGPU you need to **decorate** the Python functions that actually require a GPU with `@spaces.GPU`<br>
During the time when a decorated function is invoked, the Space will be attributed a GPU, and it will release it upon completion of the function.<br>
Here is a practical example :

```diff
+import spaces
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(...)
pipe.to('cuda')

[email protected]
def generate(prompt):
    return pipe(prompt).images

gr.Interface(
    fn=generate,
    inputs=gr.Text(),
    outputs=gr.Gallery(),
).launch()
```

1. We first `import spaces` (importing it first might prevent some issues but is not mandatory)
2. Then we decorate the `generate` function by adding a `@spaces.GPU` line before its definition

Note that `@spaces.GPU` is effect-free and can be safely used on non-ZeroGPU environments

## Duration

If you expect your GPU function to take more than __60s__ then you need to specify a `duration` param in the decorator like:

```python
@spaces.GPU(duration=120)
def generate(prompt):
   return pipe(prompt).images
```

It will set the maximum duration of your function call to 120s.

You can also specify a duration if you know that your function will take far less than the 60s default.

The lower the duration, the higher priority your Space visitors will have in the queue

<style>
  :root {
    --bg-color-light: rgba(255, 255, 255, 0.3);
    --bg-color-dark: rgba(0, 0, 0, 0.3);
    --text-color-light: rgb(0, 0, 0);
    --text-color-dark: rgb(255, 255, 255);
  }

  .theme-light {
    --bg-color: var(--bg-color-light);
    --text-color: var(--text-color-light);
  }

  .theme-dark {
    --bg-color: var(--bg-color-dark);
    --text-color: var(--text-color-dark);
  }

  .notice {
    background-color: var(--bg-color);
    color: var(--text-color);
    border-radius: 8px;
    padding: 0.5rem 1rem;
    margin-top: 10px;
  }

  .notice span {
    font-weight: 600;
  }
</style>