lemonilia commited on
Commit
517e367
1 Parent(s): 10c3982

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -7,8 +7,8 @@ license: apache-2.0
7
  This is an experimental version of LimaRP for [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) with
8
  about 1800 training samples _up to_ 4k tokens length. Contrarily to the previously released "v3" version for Llama-2, this one does
9
  not include a preliminary finetuning pass on several thousands short stories. Initial testing has shown Mistral to be capable of
10
- generating on its own the kind of stories that were included there; its training data appears to be quite diverse and not have
11
- been filtered heavily.
12
 
13
  Due to software limitations, finetuning didn't take advantage yet of the Sliding Window Attention (SWA) which would have allowed
14
  to use longer conversations in the training data. Thus, this version of LimaRP should be considered an initial attempt and
 
7
  This is an experimental version of LimaRP for [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) with
8
  about 1800 training samples _up to_ 4k tokens length. Contrarily to the previously released "v3" version for Llama-2, this one does
9
  not include a preliminary finetuning pass on several thousands short stories. Initial testing has shown Mistral to be capable of
10
+ generating on its own the kind of stories that were included there; its training data appears to be quite diverse and does not
11
+ seem to have been filtered for content type.
12
 
13
  Due to software limitations, finetuning didn't take advantage yet of the Sliding Window Attention (SWA) which would have allowed
14
  to use longer conversations in the training data. Thus, this version of LimaRP should be considered an initial attempt and