Add examples
Browse files
README.md
CHANGED
@@ -18,6 +18,37 @@ A model for [Pilota](https://github.com/megagonlabs/pilota) trained with [Accomm
|
|
18 |
- Fine tuned model of [LINE DistilBERT Japanese](https://huggingface.co/line-corporation/line-distilbert-base-japanese)
|
19 |
- The original model is distributed in [the Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)
|
20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
21 |
## License
|
22 |
|
23 |
Apache License 2.0
|
|
|
18 |
- Fine tuned model of [LINE DistilBERT Japanese](https://huggingface.co/line-corporation/line-distilbert-base-japanese)
|
19 |
- The original model is distributed in [the Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)
|
20 |
|
21 |
+
## Usage
|
22 |
+
|
23 |
+
1. Install [Pilota](https://github.com/megagonlabs/pilota)
|
24 |
+
2. Prepare inputs
|
25 |
+
- Command
|
26 |
+
|
27 |
+
```bash
|
28 |
+
echo -e 'ใ่ฆๆใใ็ฅใใใใ ใใ\tใฏใใ้จๅฑใใๅฏๅฃซๅฑฑใ่ฆใใฆใๅคๆฏใ่ฆใชใใ้ฃไบใฎใงใใใใใซใใใใชใ\nใใใซใกใฏ\tใใใซใกใฏ' | python -m pilota.convert.plain2request | tee input.jsonl
|
29 |
+
```
|
30 |
+
|
31 |
+
- Output
|
32 |
+
|
33 |
+
```jsonl
|
34 |
+
{"context": [{"name": "agent", "text": "ใ่ฆๆใใ็ฅใใใใ ใใ"}], "utterance": "ใฏใใ้จๅฑใใๅฏๅฃซๅฑฑใ่ฆใใฆใๅคๆฏใ่ฆใชใใ้ฃไบใฎใงใใใใใซใใใใชใ", "sentences": null, "meta": {}}
|
35 |
+
{"context": [{"name": "agent", "text": "ใใใซใกใฏ"}], "utterance": "ใใใซใกใฏ", "sentences": null, "meta": {}}
|
36 |
+
```
|
37 |
+
|
38 |
+
3. Feed it to Pilota
|
39 |
+
- Command
|
40 |
+
|
41 |
+
```console
|
42 |
+
pilota -m megagonlabs/pilota_dialog --batch_size 1 --outlen 60 --nbest 1 --beam 5 < input.jsonl
|
43 |
+
```
|
44 |
+
|
45 |
+
- Output
|
46 |
+
|
47 |
+
```jsonl
|
48 |
+
[{"scuds_nbest": [[]], "original_ranks": [0], "scores": [0.9911208689212798], "scores_detail": [{"OK": 0.9704028964042664, "incorrect_none": 0.04205145686864853, "lack": 0.0007874675211496651, "limited": 0.0003119863977190107, "non_fluent": 0.0002362923405598849, "untruth": 0.0013080810895189643}], "sentence": "ใฏใใ"}, {"scuds_nbest": [["้จๅฑใใๅฏๅฃซๅฑฑใ่ฆใใใใใซใ่ฏใใ", "ๅคๆฏใ่ฆใชใใ้ฃไบใฎใงใใใใใซใ่ฏใใ"]], "original_ranks": [0], "scores": [0.9952289938926696], "scores_detail": [{"OK": 0.9840966463088989, "incorrect_none": 0.010280555114150047, "lack": 0.0032871251460164785, "limited": 0.00041511686868034303, "non_fluent": 0.0002954243100248277, "untruth": 0.003289491171017289}], "sentence": "้จๅฑใใๅฏๅฃซๅฑฑใ่ฆใใฆใๅคๆฏใ่ฆใชใใ้ฃไบใฎใงใใใใใซใใใใชใ"}]
|
49 |
+
[{"scuds_nbest": [[]], "original_ranks": [0], "scores": [0.9831213414669036], "scores_detail": [{"OK": 0.9704028964042664, "incorrect_none": 0.04205145686864853, "lack": 0.0007874675211496651, "limited": 0.0003119863977190107, "non_fluent": 0.0002362923405598849, "untruth": 0.0013080810895189643}], "sentence": "ใใใซใกใฏ"}]
|
50 |
+
```
|
51 |
+
|
52 |
## License
|
53 |
|
54 |
Apache License 2.0
|