0xOracle commited on
Commit
fed0e01
1 Parent(s): 9dd414d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +41 -1
README.md CHANGED
@@ -17,4 +17,44 @@ library_name: transformers
17
  pipeline_tag: text-generation
18
  ---
19
 
20
- # Model Trained Using AutoTrain
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  pipeline_tag: text-generation
18
  ---
19
 
20
+ # Model Trained Using AutoTrain
21
+
22
+ The mistral-7b-fraud2-finetuned Large Language Model (LLM) is a fine-tuned version of the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) generative text model using a variety of synthetically generated Fraudulent transcripts datasets.
23
+
24
+ For full details of this model please read [release blog post](https://mistral.ai/news/announcing-mistral-7b/)
25
+
26
+ ## Instruction format
27
+
28
+ In order to leverage instruction fine-tuning, your prompt should be surrounded by `[INST]` and `[\INST]` tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id.
29
+
30
+ E.g.
31
+
32
+ ```python
33
+ from transformers import AutoModelForCausalLM, AutoTokenizer
34
+
35
+ device = "cuda" # the device to load the model onto
36
+
37
+ model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
38
+ tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
39
+
40
+ text = "<s>[INST] What is your favourite condiment? [/INST]"
41
+ "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
42
+ "[INST] Do you have mayonnaise recipes? [/INST]"
43
+
44
+ encodeds = tokenizer(text, return_tensors="pt", add_special_tokens=False)
45
+
46
+ model_inputs = encodeds.to(device)
47
+ model.to(device)
48
+
49
+ generated_ids = model.generate(**model_inputs, max_new_tokens=1000, do_sample=True)
50
+ decoded = tokenizer.batch_decode(generated_ids)
51
+ print(decoded[0])
52
+ ```
53
+
54
+ ## Model Architecture
55
+ This instruction model is based on Mistral-7B-v0.1, a transformer model with the following architecture choices:
56
+ - Grouped-Query Attention
57
+ - Sliding-Window Attention
58
+ - Byte-fallback BPE tokenizer
59
+
60
+ ## The Team