jpacifico commited on
Commit
6e68e4c
·
verified ·
1 Parent(s): e453b4f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +51 -7
README.md CHANGED
@@ -5,25 +5,69 @@ language:
5
  - fr
6
  - en
7
  tags:
8
- - phi-3
9
  - french
10
- - phi-3-3B
11
  - chocolatine
 
 
 
12
  ---
13
 
14
- ### Model Description
15
 
16
- Chocolatine-3B-Instruct-DPO-Revised
17
- 3.82B params. Window context = 4k tokens
18
- DPO fine-tuned of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
19
  using the [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised) rlhf dataset.
20
  Chocolatine is a general model and can itself be finetuned to be specialized for specific use cases.
 
21
 
22
  ![image/jpeg](https://github.com/jpacifico/Chocolatine-LLM/blob/main/Assets/chocolatine_visuel_500x500.png?raw=true)
23
 
24
  ### Evaluation
25
 
26
- Benchmarks very soon ^^
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
 
28
  ### Limitations
29
 
 
5
  - fr
6
  - en
7
  tags:
 
8
  - french
 
9
  - chocolatine
10
+ datasets:
11
+ - jpacifico/french-orca-dpo-pairs-revised
12
+ pipeline_tag: text-generation
13
  ---
14
 
15
+ ### Chocolatine-3B-Instruct-DPO-Revised
16
 
17
+ DPO fine-tuned of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) (3.82B params)
 
 
18
  using the [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised) rlhf dataset.
19
  Chocolatine is a general model and can itself be finetuned to be specialized for specific use cases.
20
+ Window context = 4k tokens
21
 
22
  ![image/jpeg](https://github.com/jpacifico/Chocolatine-LLM/blob/main/Assets/chocolatine_visuel_500x500.png?raw=true)
23
 
24
  ### Evaluation
25
 
26
+ Submitted on [OpenLLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard), results in few days !
27
+
28
+ ### Evaluation in French
29
+
30
+ Chocolatine-3B-Instruct-DPO-Revised is outperforming GPT-3.5-Turbo on [mt-bench-french](https://huggingface.co/datasets/bofenghuang/mt-bench-french) by Bofeng Huang,
31
+ used with [multilingual-mt-bench](https://github.com/Peter-Devine/multilingual_mt_bench)
32
+
33
+ ![image/jpeg](https://github.com/jpacifico/Chocolatine-LLM/blob/main/Assets/results_fr_mt_bench_400x900.png?raw=false)
34
+
35
+ ### Usage
36
+
37
+ You can run this model my [Colab notebook](https://github.com/jpacifico/Chocolatine-LLM/blob/main/Chocolatine_3B_inference_test_colab.ipynb)
38
+
39
+ You can also run this model using the following code:
40
+
41
+ ```python
42
+ import transformers
43
+ from transformers import AutoTokenizer
44
+
45
+ # Format prompt
46
+ message = [
47
+ {"role": "system", "content": "You are a helpful assistant chatbot."},
48
+ {"role": "user", "content": "What is a Large Language Model?"}
49
+ ]
50
+ tokenizer = AutoTokenizer.from_pretrained(new_model)
51
+ prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)
52
+
53
+ # Create pipeline
54
+ pipeline = transformers.pipeline(
55
+ "text-generation",
56
+ model=new_model,
57
+ tokenizer=tokenizer
58
+ )
59
+
60
+ # Generate text
61
+ sequences = pipeline(
62
+ prompt,
63
+ do_sample=True,
64
+ temperature=0.7,
65
+ top_p=0.9,
66
+ num_return_sequences=1,
67
+ max_length=200,
68
+ )
69
+ print(sequences[0]['generated_text'])
70
+ ```
71
 
72
  ### Limitations
73