qgallouedec HF Staff commited on
Commit
6f3588c
·
verified ·
1 Parent(s): 31a78c4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +155 -1
README.md CHANGED
@@ -7,4 +7,158 @@ sdk: docker
7
  pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  pinned: false
8
  ---
9
 
10
+ # Getting Started with the Jobs API
11
+
12
+ The Jobs API allows users to execute scripts remotely. The key concept is simple: you write a script, host it in a dedicated space, and then any user can execute it as a job with their arguments. Let's walk through the setup step by step.
13
+
14
+ ## Creating a Space
15
+
16
+ To begin, you'll need to create a space to host your script. Follow these steps:
17
+
18
+ 1. Visit [Hugging Face Spaces](https://huggingface.co/spaces).
19
+ 2. Click on `New Space`.
20
+ 3. Name your space (e.g., `run-hello-world`).
21
+ 4. Select `Docker` as the space type.
22
+ 5. Choose the `Blank` template.
23
+ 6. For hardware, select `CPU basic • 2vCPU • 16GB FREE` (note: this does not limit the resources used when running jobs).
24
+ 7. Set visibility to `Public`.
25
+
26
+ Once created, your space will contain two default files: `README.md` and `.gitattributes`. To make it functional for running jobs, you need to add two more files: `Dockerfile` and `run_job.py`.
27
+
28
+ ### Setting Up the Environment
29
+
30
+ The `Dockerfile` defines the environment in which your script will run. Below is the simplest version to get started:
31
+
32
+ ```dockerfile
33
+ # Use a lightweight Python image as the base
34
+ FROM python:3.11-slim
35
+
36
+ # Set the working directory inside the container
37
+ WORKDIR /app
38
+
39
+ # Copy all files to the container
40
+ COPY . /app
41
+ ```
42
+
43
+ ### Writing the Job Script
44
+
45
+ The second required file is `run_job.py`, which contains the script you want to execute. To start, create a simple script that prints a message:
46
+
47
+ ```python
48
+ print("Hello, world!")
49
+ ```
50
+
51
+ At this stage, your space will display a `Runtime error`. This is expected since no application has been defined yet. However, this is not an issue as we don’t need one for job execution. You can add an application later if you want to provide a user-friendly interface for managing jobs.
52
+
53
+ ## Running a Job
54
+
55
+ ### Step 1: Create an API Token
56
+
57
+ Before launching a job, you need to create a dedicated token for job execution. This token identifies the user requesting the job, ensures access permissions, and handles billing if necessary.
58
+
59
+ 1. Go to [Hugging Face Tokens](https://huggingface.co/settings/tokens).
60
+ 2. Click on `Create new token`.
61
+ 3. Give it a name of your choice.
62
+ 4. Select the permission `Start and manage Jobs on your behalf`.
63
+ 5. Click `Create token`.
64
+ 6. Copy the token and store it securely. Do not share it with anyone.
65
+
66
+ ### Step 2: Execute a Job
67
+
68
+ Open a terminal and run the following command:
69
+
70
+ ```bash
71
+ curl -L 'https://huggingface.co/api/jobs/<username>' \
72
+ -H 'Content-Type: application/json' \
73
+ -H 'Authorization: Bearer <token>' \
74
+ -d '{
75
+ "spaceId": "<space-id>",
76
+ "command": ["python", "<script-name>.py"],
77
+ "arguments": ["--arg1", "value1", "--arg2", "value2"],
78
+ "environment": {"ENV_VAR1": "value1", "ENV_VAR2": "value2"},
79
+ "flavor": "<flavor>"
80
+ }'
81
+ ```
82
+
83
+ Let's break down the command:
84
+
85
+ - `-L 'https://huggingface.co/api/jobs/<username>'`: API endpoint to launch a job. Replace `<username>` with your Hugging Face username.
86
+ - `-H 'Authorization: Bearer <token>'`: The token you created earlier, used for authentication and permission verification.
87
+ - `-d '{...}'`: The body of the request, containing job parameters:
88
+ - `spaceId`: The identifier of your space (e.g., `qgallouedec/run-hello-world`).
89
+ - `command`: The command to execute. Here, we run a Python script: `["python", "run_job.py"]`.
90
+ - `arguments`: Optional arguments for the script. If none are needed, use an empty array: `[]`.
91
+ - `environment`: Environment variables for the job. If none are needed, use `{}`.
92
+ - `flavor`: The type of machine to run the job on. Available options include `cpu-basic`, `cpu-upgrade`, `t4-small`, `t4-medium`, `a10g-small`, `a10g-large`, `a10g-largex2`, `a10glargex4`, `a100-large`, `h100`, `h100x8`, `l4x4`, `l4x1`, `l40sx1`, `l40sx4`, `l40sx8`.
93
+
94
+ In my example, the command would look like this:
95
+
96
+ ```bash
97
+ curl -L 'https://huggingface.co/api/jobs/qgallouedec' \
98
+ -H 'Content-Type: application/json' \
99
+ -H 'Authorization: Bearer hf_MyTokenIsSecret' \
100
+ -d '{
101
+ "spaceId": "qgallouedec/run-hello-world",
102
+ "command": ["python", "run_job.py"],
103
+ "arguments": [],
104
+ "environment": {},
105
+ "flavor": "cpu-basic"
106
+ }'
107
+ ```
108
+
109
+ In the above example, the namespace of the space (`qgallouedec`) matches the username of the person who launch the job. However, this is usually not always the case, as users can launch jobs from spaces created by others.
110
+
111
+ ### Step 3: Interpreting the Response
112
+
113
+ Once the job is launched, you'll receive a response like this:
114
+
115
+ ```json
116
+ {
117
+ "metadata": {
118
+ "job_id": "-FZNr5FTonBJJ8F2ENXnDQ5SY",
119
+ "owner": {"id": "631ce4b244503b72277fc89f", "name": "qgallouedec"}
120
+ },
121
+ "spec": {
122
+ "input": {"spaceId": "qgallouedec/run-hello-world"},
123
+ "timeout": null,
124
+ "env": {},
125
+ "command": ["python", "run_job.py"],
126
+ "args": [],
127
+ "flavor": "cpu-basic",
128
+ "owner": {"id": "631ce4b244503b72277fc89f", "name": "qgallouedec"}
129
+ },
130
+ "status": {
131
+ "stage": "RUNNING",
132
+ "error": null,
133
+ "message": null,
134
+ "abuse": {"flagged": false, "reason": "", "detector": "", "at": "2025-03-17T00:22:06.432Z"}
135
+ }
136
+ }
137
+ ```
138
+
139
+ The most important field here is `job_id` (`-FZNr5FTonBJJ8F2ENXnDQ5SY`), which allows you to track the job's progress and access logs to verify execution.
140
+
141
+ ## Monitoring Job Progress
142
+
143
+ To monitor the job's progress, you can use the following command:
144
+
145
+ ```bash
146
+ curl -L 'https://huggingface.co/api/jobs/<username>/<job_id>/logs-stream' \
147
+ -H 'Authorization: Bearer <token>'
148
+ ```
149
+
150
+ In my case, the command would look like this:
151
+
152
+ ```bash
153
+ curl -L 'https://huggingface.co/api/jobs/qgallouedec/-FZNr5FTonBJJ8F2ENXnDQ5SY/logs-stream' \
154
+ -H 'Authorization: Bearer hf_MyTokenIsSecret'
155
+ ```
156
+
157
+ And you'll receive a stream of logs like this:
158
+
159
+ ```txt
160
+ data: {"data":"===== Job started at 2025-03-17 00:22:06 =====","timestamp":"2025-03-17T00:22:06Z"}
161
+
162
+ data: {"data":"Hello, world!","timestamp":"2025-03-17T00:22:08.836Z"}
163
+ ```
164
+