Xenova HF staff commited on
Commit
0343bfb
·
1 Parent(s): ece9ad5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +66 -0
README.md CHANGED
@@ -6,4 +6,70 @@ tags:
6
 
7
  https://huggingface.co/laion/clap-htsat-unfused with ONNX weights to be compatible with Transformers.js.
8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).
 
6
 
7
  https://huggingface.co/laion/clap-htsat-unfused with ONNX weights to be compatible with Transformers.js.
8
 
9
+ ## Usage (Transformers.js)
10
+
11
+ If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@xenova/transformers) using:
12
+ ```bash
13
+ npm i @xenova/transformers
14
+ ```
15
+
16
+ **Example:** Perform zero-shot audio classification with `Xenova/clap-htsat-unfused`.
17
+ ```js
18
+ let classifier = await pipeline('zero-shot-audio-classification', 'Xenova/clap-htsat-unfused');
19
+ let audio = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/dog_barking.wav';
20
+ let candidate_labels = ['dog', 'vaccum cleaner'];
21
+ let scores = await classifier(audio, candidate_labels);
22
+ // [
23
+ // { score: 0.9993992447853088, label: 'dog' },
24
+ // { score: 0.0006007603369653225, label: 'vaccum cleaner' }
25
+ // ]
26
+ ```
27
+
28
+ **Example:** Compute text embeddings with `ClapTextModelWithProjection`.
29
+
30
+ ```js
31
+ import { AutoTokenizer, ClapTextModelWithProjection } from '@xenova/transformers';
32
+
33
+ // Load tokenizer and text model
34
+ const tokenizer = await AutoTokenizer.from_pretrained('Xenova/clap-htsat-unfused');
35
+ const text_model = await ClapTextModelWithProjection.from_pretrained('Xenova/clap-htsat-unfused');
36
+
37
+ // Run tokenization
38
+ const texts = ['a sound of a cat', 'a sound of a dog'];
39
+ const text_inputs = tokenizer(texts, { padding: true, truncation: true });
40
+
41
+ // Compute embeddings
42
+ const { text_embeds } = await text_model(text_inputs);
43
+ // Tensor {
44
+ // dims: [ 2, 512 ],
45
+ // type: 'float32',
46
+ // data: Float32Array(1024) [ ... ],
47
+ // size: 1024
48
+ // }
49
+ ```
50
+
51
+ **Example:** Compute audio embeddings with `ClapAudioModelWithProjection`.
52
+ ```js
53
+ import { AutoProcessor, ClapAudioModelWithProjection, read_audio } from '@xenova/transformers';
54
+
55
+ // Load processor and audio model
56
+ const processor = await AutoProcessor.from_pretrained('Xenova/clap-htsat-unfused');
57
+ const audio_model = await ClapAudioModelWithProjection.from_pretrained('Xenova/clap-htsat-unfused');
58
+
59
+ // Read audio and run processor
60
+ const audio = await read_audio('https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/cat_meow.wav');
61
+ const audio_inputs = await processor(audio);
62
+
63
+ // Compute embeddings
64
+ const { audio_embeds } = await audio_model(audio_inputs);
65
+ // Tensor {
66
+ // dims: [ 1, 512 ],
67
+ // type: 'float32',
68
+ // data: Float32Array(512) [ ... ],
69
+ // size: 512
70
+ // }
71
+ ```
72
+
73
+ ---
74
+
75
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).