Update README.md
Browse files
README.md
CHANGED
@@ -164,7 +164,7 @@ pip install -r requirements.txt
|
|
164 |
```
|
165 |
|
166 |
### **Overview of `clamp3_*.py` Scripts**
|
167 |
-
CLaMP 3 provides scripts for **semantic
|
168 |
|
169 |
Supported formats include:
|
170 |
- **Audio**: `.mp3`, `.wav`
|
@@ -179,6 +179,14 @@ Supported formats include:
|
|
179 |
|
180 |
> **Note**: All files in a folder must belong to the same modality for processing.
|
181 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
182 |
#### **[`clamp3_score.py`](https://github.com/sanderwood/clamp3/blob/main/clamp3_score.py) - Semantic Similarity Calculation**
|
183 |
|
184 |
This script calculates semantic similarity between query and reference files. By default, it uses **pairwise mode**, but you can switch to **group mode** using the `--group` flag.
|
@@ -223,22 +231,26 @@ python clamp3_score.py <query_dir> <ref_dir> [--group]
|
|
223 |
python clamp3_score.py query_dir ref_dir --group
|
224 |
```
|
225 |
|
226 |
-
#### **[`
|
227 |
|
228 |
-
|
229 |
|
230 |
```bash
|
231 |
-
python
|
232 |
```
|
233 |
|
234 |
-
#### **[`
|
235 |
|
236 |
-
|
237 |
|
238 |
```bash
|
239 |
-
python
|
240 |
```
|
241 |
|
|
|
|
|
|
|
|
|
242 |
## **Repository Structure**
|
243 |
- **[code/](https://github.com/sanderwood/clamp3/tree/main/code)** → Training & feature extraction scripts.
|
244 |
- **[classification/](https://github.com/sanderwood/clamp3/tree/main/classification)** → Linear classification training and prediction.
|
|
|
164 |
```
|
165 |
|
166 |
### **Overview of `clamp3_*.py` Scripts**
|
167 |
+
CLaMP 3 provides scripts for **semantic search**, **semantic similarity calculation**, **retrieval performance evaluation**, and **feature extraction** across five modalities. Simply provide the file path, and the script will automatically detect the modality and extract the relevant features.
|
168 |
|
169 |
Supported formats include:
|
170 |
- **Audio**: `.mp3`, `.wav`
|
|
|
179 |
|
180 |
> **Note**: All files in a folder must belong to the same modality for processing.
|
181 |
|
182 |
+
#### **[`clamp3_search.py`](https://github.com/sanderwood/clamp3/blob/main/clamp3_search.py) - Semantic Search**
|
183 |
+
|
184 |
+
Run retrieval tasks by comparing a query file to reference files in `ref_dir`. The query and `ref_dir` can be **any modality**, so there are **25 possible retrieval combinations**, e.g., text-to-music, image-to-music, music-to-music, music-to-text (zero-shot music classification), etc.
|
185 |
+
|
186 |
+
```bash
|
187 |
+
python clamp3_search.py <query_file> <ref_dir> [--top_k TOP_K]
|
188 |
+
```
|
189 |
+
|
190 |
#### **[`clamp3_score.py`](https://github.com/sanderwood/clamp3/blob/main/clamp3_score.py) - Semantic Similarity Calculation**
|
191 |
|
192 |
This script calculates semantic similarity between query and reference files. By default, it uses **pairwise mode**, but you can switch to **group mode** using the `--group` flag.
|
|
|
231 |
python clamp3_score.py query_dir ref_dir --group
|
232 |
```
|
233 |
|
234 |
+
#### **[`clamp3_eval.py`](https://github.com/sanderwood/clamp3/blob/main/clamp3_eval.py) - Retrieval Performance Evaluation**
|
235 |
|
236 |
+
Evaluates **CLaMP3's retrieval performance** on a paired dataset using metrics like **MRR** and **Hit@K**. Works the same way as **pairwise mode** in `clamp3_score.py`—requiring **matching folder structure** and **filenames** between `query_dir` and `ref_dir`.
|
237 |
|
238 |
```bash
|
239 |
+
python clamp3_eval.py <query_dir> <ref_dir>
|
240 |
```
|
241 |
|
242 |
+
#### **[`clamp3_embd.py`](https://github.com/sanderwood/clamp3/blob/main/clamp3_embd.py) - Feature Extraction**
|
243 |
|
244 |
+
If other scripts don't meet your needs, use `clamp3_embd.py` to extract features.
|
245 |
|
246 |
```bash
|
247 |
+
python clamp3_embd.py <input_dir_path> <output_dir_path> [--get_global]
|
248 |
```
|
249 |
|
250 |
+
**Feature Output:**
|
251 |
+
- **Without `--get_global`** → Shape: **(1, T, 768)** (T = time steps). Uses last hidden states before avg pooling, ideal for applications needing temporal info. Fine-tuning recommended.
|
252 |
+
- **With `--get_global`** → Shape: **(1, 768)**. Uses avg pooled features, suitable for applications needing global info, can be used directly.
|
253 |
+
|
254 |
## **Repository Structure**
|
255 |
- **[code/](https://github.com/sanderwood/clamp3/tree/main/code)** → Training & feature extraction scripts.
|
256 |
- **[classification/](https://github.com/sanderwood/clamp3/tree/main/classification)** → Linear classification training and prediction.
|