speed commited on
Commit
7fd0df9
·
verified ·
1 Parent(s): 667dae1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -2
README.md CHANGED
@@ -3,6 +3,36 @@ tags:
3
  - clip
4
  library_name: open_clip
5
  pipeline_tag: zero-shot-image-classification
6
- license: mit
7
  ---
8
- # Model card for llm-jp-roberta-ViT-L-14-relaion-1.5B-lr5e-4-bs8k-accum4-20241218-epoch90
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  - clip
4
  library_name: open_clip
5
  pipeline_tag: zero-shot-image-classification
6
+ license: [apache-2.0]
7
  ---
8
+ # Model Card for llm-jp-roberta-ViT-L-14-relaion-1.5B-lr5e-4-bs8k-accum4-20241218-epoch90
9
+
10
+ # Model Details
11
+
12
+ A CLIP ViT-L/14 model trained using [OpenCLIP](https://github.com/mlfoundations/open_clip) with the Japanese translation of the English subset of ReLAION-5B (https://huggingface.co/datasets/laion/relaion2B-en-research-safe), translated by [gemma-2-9b-it](https://huggingface.co/datasets/laion/relaion2B-en-research-safe).
13
+
14
+ # How to Use
15
+
16
+
17
+ # Training Details
18
+
19
+ ## Model Architecture
20
+
21
+ - Text Encoder: RoBERTa base with llm-jp-tokenizer
22
+ - Image Encoder: ViT-L/14
23
+
24
+ ## Training Data
25
+
26
+ We used a Japanese-translated version of the relaion2B-en-research-safe dataset.
27
+ The translation was performed using gemma-2-9b-it.
28
+ Due to a 70% success rate in image downloads, the dataset size was 1.45 billion samples, and we processed it over 9 epochs (13 billion samples in total).
29
+
30
+ # Evaluation
31
+
32
+ # Citation
33
+
34
+ Bibtex:
35
+ ```
36
+ ```
37
+
38
+