gsarti commited on
Commit
15d7303
·
verified ·
1 Parent(s): 38c2fcf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +130 -9
README.md CHANGED
@@ -1,22 +1,143 @@
1
  ---
2
- base_model: unsloth/Phi-3-mini-4k-instruct-v0-bnb-4bit
3
  language:
4
- - en
5
  license: apache-2.0
 
6
  tags:
7
  - text-generation-inference
8
- - transformers
9
  - unsloth
10
  - mistral
11
  - trl
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ---
13
 
14
- # Uploaded model
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- - **Developed by:** gsarti
17
- - **License:** apache-2.0
18
- - **Finetuned from model :** unsloth/Phi-3-mini-4k-instruct-v0-bnb-4bit
19
 
20
- This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
1
  ---
 
2
  language:
3
+ - it
4
  license: apache-2.0
5
+ library_name: transformers
6
  tags:
7
  - text-generation-inference
 
8
  - unsloth
9
  - mistral
10
  - trl
11
+ - word-game
12
+ - rebus
13
+ - italian
14
+ - word-puzzle
15
+ - crossword
16
+ datasets:
17
+ - gsarti/eureka-rebus
18
+ base_model: unsloth/Phi-3-mini-4k-instruct-v0-bnb-4bit
19
+
20
+ model-index:
21
+ - name: gsarti/phi3-mini-rebus-solver-fp16
22
+ results:
23
+ - task:
24
+ type: verbalized-rebus-solving
25
+ name: Verbalized Rebus Solving
26
+ dataset:
27
+ type: gsarti/eureka-rebus
28
+ name: EurekaRebus
29
+ config: llm_sft
30
+ split: test
31
+ revision: 0f24ebc3b66cd2f8968077a5eb058be1d5af2f05
32
+ metrics:
33
+ - type: exact_match
34
+ value: 0.56
35
+ name: First Pass Exact Match
36
+ - type: exact_match
37
+ value: 0.51
38
+ name: Solution Exact Match
39
  ---
40
 
41
+ # Phi-3 Mini 4K Verbalized Rebus Solver 🇮🇹
42
+
43
+ This model is a parameter-efficient fine-tuned version of Phi-3 Mini 4K trained for verbalized rebus solving in Italian, as part of the [release](https://huggingface.co/collections/gsarti/verbalized-rebus-clic-it-2024-66ab8f11cb04e68bdf4fb028) for our paper [Non Verbis, Sed Rebus: Large Language Models are Weak Solvers of Italian Rebuses](TBD). The task of verbalized rebus solving consists of converting an encrypted sequence of letters and crossword definitions into a solution phrase matching the word lengths specified in the solution key. An example is provided below.
44
+
45
+ The model was trained in 4-bit precision for 5070 steps on the verbalized subset of the [EurekaRebus](https://huggingface.co/datasets/gsarti/eureka-rebus) using QLora via [Unsloth](https://github.com/unslothai/unsloth) and [TRL](https://github.com/huggingface/trl). This version has merged adapter weights in half precision, enabling out-of-the-box for usage with the `transformers` library.
46
+
47
+ We also provide [adapter checkpoints through training](gsarti/phi3-mini-rebus-solver-adapters) and [8-bit GGUF](gsarti/phi3-mini-rebus-solver-Q8_0-GGUF) versions of this model for analysis and local execution.
48
+
49
+ ## Using the Model
50
+
51
+ The following example shows how to perform inference using Unsloth or Transformers:
52
+
53
+ ```python
54
+
55
+ # With Unsloth (efficient, requires GPU)
56
+ from unsloth import FastLanguageModel
57
+
58
+ model, tokenizer = FastLanguageModel.from_pretrained(
59
+ model_name = "gsarti/phi3-mini-rebus-solver-fp16",
60
+ max_seq_length = 1248,
61
+ load_in_4bit = True,
62
+ )
63
+
64
+ # Or with Transformers
65
+ from transformers import AutoTokenizer, AutoModelForCausalLM
66
+
67
+ tokenizer = AutoTokenizer.from_pretrained("gsarti/phi3-mini-rebus-solver-fp16")
68
+ model = AutoModelForCausalLM.from_pretrained("gsarti/phi3-mini-rebus-solver-fp16")
69
+
70
+ # Inference
71
+ verbalized_rebus = "[Materiale espulso dai vulcani] R O [Strumento del calzolaio] [Si trovano ai lati del bacino] C I [Si ingrassano con la polenta] E I N [Contiene scorte di cibi] B [Isola in francese]"
72
+ solution_key = "1 ' 5 6 5 3 3 1 14"
73
+ template = """<s><|user|>
74
+ Risolvi gli indizi tra parentesi per ottenere una prima lettura, e usa la chiave di lettura per ottenere la soluzione del rebus.
75
+
76
+ Rebus: {rebus}
77
+ Chiave risolutiva: {key}<|end|>
78
+ <|assistant|>"""
79
+ input = template.format(rebus=verbalized_rebus, key=solution_key)
80
+ inputs = tokenizer(input, return_tensors="pt")["input_ids"]
81
+ outputs = model.generate(input_ids = inputs, max_new_tokens = 500, use_cache = True)
82
+ model_generations = tokenizer.batch_decode(outputs)
83
+ print(model_generations[0])
84
+
85
+ # Procediamo alla risoluzione del rebus passo per passo:
86
+ # - [Materiale espulso dai vulcani] = lava
87
+ # - R O = R O
88
+ # - [Strumento del calzolaio] = lesina
89
+ # - [Si trovano ai lati del bacino] = anche
90
+ # - C I = C I
91
+ # - [Si ingrassano con la polenta] = oche
92
+ # - E I N = E I N
93
+ # - [Contiene scorte di cibi] = silos
94
+ # - B = B
95
+ # - [Isola in francese] = ile
96
+ #
97
+ # Prima lettura: lava R O lesina anche C I oche E I N silos B ile
98
+ #
99
+ # Ora componiamo la soluzione seguendo la chiave risolutiva:
100
+ # 1 = L
101
+ # ' = '
102
+ # 5 = avaro
103
+ # 6 = lesina
104
+ # 5 = anche
105
+ # 3 = ciò
106
+ # 3 = che
107
+ # 1 = è
108
+ # 14 = insilosbile
109
+ #
110
+ # Soluzione: L'avaro lesina anche ciò che è insilosbile
111
+ ```
112
+
113
+ See the official [code release](https://github.com/gsarti/verbalized-rebus) for more examples.
114
+
115
+ ### Local usage with Ollama
116
+
117
+ A ready-to-use local version of this model is hosted on the [Ollama Hub](https://ollama.com/gsarti/phi3-mini-rebus-solver) and can be used as follows:
118
+
119
+ ```shell
120
+ ollama run gsarti/phi3-mini-rebus-solver "Rebus: [Materiale espulso dai vulcani] R O [Strumento del calzolaio] [Si trovano ai lati del bacino] C I [Si ingrassano con la polenta] E I N [Contiene scorte di cibi] B [Isola in francese]\nChiave risolutiva: 1 ' 5 6 5 3 3 1 14"
121
+ ```
122
+
123
+ ## Limitations
124
+
125
+ **Lexical overfitting**: As remarked in the related publication, the model overfitted the set of definitions/answers for first pass words. As a result, words that were [explicitly witheld](https://huggingface.co/datasets/gsarti/eureka-rebus/blob/main/ood_words.txt) from the training set cause significant performance degradation when used as solutions for verbalized rebuses' definitions. You can compare model performances between [in-domain](https://huggingface.co/datasets/gsarti/eureka-rebus/blob/main/id_test.jsonl) and [out-of-domain](https://huggingface.co/datasets/gsarti/eureka-rebus/blob/main/ood_test.jsonl) test examples to verify this limitation.
126
+
127
+ ## Model curators
128
+
129
+ For problems or updates on this model, please contact [[email protected]](mailto:[email protected]).
130
+
131
+ ### Citation Information
132
+
133
+ If you use this model in your work, please cite our paper as follows:
134
+
135
+ ```bibtex
136
+ TBD
137
+ ```
138
 
139
+ ## Acknowledgements
 
 
140
 
141
+ We are grateful to the [Associazione Culturale "Biblioteca Enigmistica Italiana - G. Panini"](http://www.enignet.it/home) for making its rebus collection freely accessible on the [Eureka5 platform](http://www.eureka5.it).
142
 
143
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)