asofter commited on
Commit
f76fe30
·
1 Parent(s): 3a0e12b

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +76 -0
README.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - multilingual
4
+ - ar
5
+ - bg
6
+ - de
7
+ - el
8
+ - en
9
+ - es
10
+ - fr
11
+ - hi
12
+ - it
13
+ - ja
14
+ - nl
15
+ - pl
16
+ - pt
17
+ - ru
18
+ - sw
19
+ - th
20
+ - tr
21
+ - ur
22
+ - vi
23
+ - zh
24
+ license: mit
25
+ inference: false
26
+ tags:
27
+ - language
28
+ - language-detection
29
+ metrics:
30
+ - accuracy
31
+ - f1
32
+ base_model: xlm-roberta-base
33
+ model-index:
34
+ - name: xlm-roberta-base-language-detection
35
+ results: []
36
+ pipeline_tag: text-classification
37
+ ---
38
+
39
+ # ONNX version of papluca/xlm-roberta-base-language-detection
40
+
41
+ **This model is a conversion of [papluca/xlm-roberta-base-language-detection](https://huggingface.co/papluca/xlm-roberta-base-language-detection) to ONNX** format using the [🤗 Optimum](https://huggingface.co/docs/optimum/index) library.
42
+
43
+ ## Model description
44
+
45
+ This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the [Language Identification](https://huggingface.co/datasets/papluca/language-identification#additional-information) dataset.
46
+
47
+ This model is an XLM-RoBERTa transformer model with a classification head on top (i.e. a linear layer on top of the pooled output).
48
+ For additional information please refer to the [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) model card or to the paper [Unsupervised Cross-lingual Representation Learning at Scale](https://arxiv.org/abs/1911.02116) by Conneau et al.
49
+
50
+ ## Intended uses & limitations
51
+
52
+ You can directly use this model as a language detector, i.e. for sequence classification tasks. Currently, it supports the following 20 languages:
53
+
54
+ `arabic (ar), bulgarian (bg), german (de), modern greek (el), english (en), spanish (es), french (fr), hindi (hi), italian (it), japanese (ja), dutch (nl), polish (pl), portuguese (pt), russian (ru), swahili (sw), thai (th), turkish (tr), urdu (ur), vietnamese (vi), and chinese (zh)`
55
+
56
+ ## Usage
57
+
58
+ Loading the model requires the [🤗 Optimum](https://huggingface.co/docs/optimum/index) library installed.
59
+
60
+ ```python
61
+ from optimum.onnxruntime import ORTModelForSequenceClassification
62
+ from transformers import AutoTokenizer, pipeline
63
+
64
+
65
+ tokenizer = AutoTokenizer.from_pretrained("laiyer/xlm-roberta-base-language-detection-onnx")
66
+ model = ORTModelForSequenceClassification.from_pretrained("laiyer/xlm-roberta-base-language-detection-onnx")
67
+ classifier = pipeline(
68
+ task="text-classification",
69
+ model=model,
70
+ tokenizer=tokenizer,
71
+ top_k=None,
72
+ )
73
+
74
+ classifier_output = ner("It's not toxic comment")
75
+ print(classifier_output)
76
+ ```