Hugging Face's logo

lst-nectec
/
HoogBERTa

Fill-Mask
Transformers
PyTorch
Safetensors
Thai
roberta
Inference Endpoints
Model card Files Community
2
HoogBERTa
File size: 2,562 Bytes 
e4b1b40
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
 
---
license: mit
datasets:
- scb_mt_enth_2020
- oscar
- best2009
- wikipedia
language:
- th
library_name: fairseq
---
# HoogBERTa

This repository includes the Thai pretrained language representation (HoogBERTa_base) and the fine-tuned model for multitask sequence labeling.



# Documentation

To initialize the model from hub, use the following commands
```
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("new5558/HoogBERTa")
model = AutoModel.from_pretrained("new5558/HoogBERTa")
```

To annotate POS, NE and cluase boundary, use the following commands
```

```

To extract token features, based on the RoBERTa architecture, use the following commands

```python
with torch.no_grad():
    model.eval()
    sentence = "วันที่ 12 มีนาคมนี้ ฉันจะไปเที่ยววัดพระแก้ว ที่กรุงเทพ"
    all_sent = []
    sentences = sentence.split(" ")
    for sent in sentences:
        all_sent.append(" ".join(tokenize(sent)).replace("_","[!und:]"))

    sentence = " _ ".join(all_sent)
    token_ids = tokenizer(sentence, return_tensors = 'pt')['input_ids']
    features = model(token_ids)
```

For batch processing,

```python
with torch.no_grad():
    model.eval()
    sentenceL = ["วันที่ 12 มีนาคมนี้","ฉันจะไปเที่ยววัดพระแก้ว ที่กรุงเทพ"]
    inputList = []
    for sentX in sentenceL:
        sentences = sentX.split(" ")
        all_sent = []
        for sent in sentences:
            all_sent.append(" ".join(tokenize(sent)).replace("_","[!und:]"))

        sentence = " _ ".join(all_sent)
        inputList.append(sentence)
    token_ids = tokenizer(inputList, padding = True, return_tensors = 'pt').input_ids
    features = model(token_ids)
```

To use HoogBERTa as an embedding layer, use

```python
with torch.no_grad():
  features = model(token_ids) # where token_ids is a tensor with type "long".
```

# Citation

Please cite as:

``` bibtex
@inproceedings{porkaew2021hoogberta,
  title = {HoogBERTa: Multi-task Sequence Labeling using Thai Pretrained Language Representation},
  author = {Peerachet Porkaew, Prachya Boonkwan and Thepchai Supnithi},
  booktitle = {The Joint International Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP 2021)},
  year = {2021},
  address={Online}
}
```

Download full-text [PDF](https://drive.google.com/file/d/1hwdyIssR5U_knhPE2HJigrc0rlkqWeLF/view?usp=sharing)