Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,39 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
+
<h1 align="center">UForm</h1>
|
5 |
+
<h3 align="center">
|
6 |
+
Multi-Modal Inference Library<br/>
|
7 |
+
For Semantic Search Applications<br/>
|
8 |
+
</h3>
|
9 |
+
|
10 |
+
---
|
11 |
+
|
12 |
+
UForm is a Multi-Modal Modal Inference package, designed to encode Multi-Lingual Texts, Images, and, soon, Audio, Video, and Documents, into a shared vector space!
|
13 |
+
|
14 |
+
This is the repository of [English](https://huggingface.co/unum-cloud/uform-vl-english/tree/main) and [multilingual](https://huggingface.co/unum-cloud/uform-vl-multilingual) UForm models converted to CoreML MLProgram format.
|
15 |
+
Currently, only unimodal parts of models are converted.
|
16 |
+
|
17 |
+
## Descriptions
|
18 |
+
|
19 |
+
Each model is separated into two parts: `image-encoder` and `text-encoder`:
|
20 |
+
|
21 |
+
* English image-encoder: [english.image-encoder.mlpackage](https://huggingface.co/unum-cloud/uform-coreml/blob/main/english.image-encoder.mlpackage.zip)
|
22 |
+
* English text-encoder: [english.text-encoder.mlpackage](https://huggingface.co/unum-cloud/uform-coreml/blob/main/english.text-encoder.mlpackage.zip)
|
23 |
+
* Multilingual image-encoder: [multilingual.image-encoder.mlpackage](https://huggingface.co/unum-cloud/uform-coreml/blob/main/multilingual.image-encoder.mlpackage.zip)
|
24 |
+
* Multilingual text-encoder: [multilingual.text-encoder.mlpackage](https://huggingface.co/unum-cloud/uform-coreml/blob/main/multilingual.text-encoder.mlpackage.zip)
|
25 |
+
|
26 |
+
Each checkpoint is a zip archive with an MLProgram of the corresponding encoder.
|
27 |
+
|
28 |
+
A text encoder has the following input fields:
|
29 |
+
|
30 |
+
* `input_ids`: int32
|
31 |
+
* `attention_mask`: int32
|
32 |
+
|
33 |
+
An image encoder has a single input field `image`: float32
|
34 |
+
|
35 |
+
Both encoders return:
|
36 |
+
|
37 |
+
* `features`: float32
|
38 |
+
* `embeddings`: float32
|
39 |
+
|