Issue with repetition
I'm getting some severe repetition after 4-5k context where it would eventually get into endless loops of a sentence, or a few sentences.
Using the linked master import settings in ST.
I have the same problem, but I am able to work around it a bit by editing the response when it tries to repeat itself. I also use the master import.
Try dropping min-p to 0.03 instead of 0.1 And upping temp to 0.95 from 0.85.
I also use the master import, immediatelly sets the temp to 1 and didn't face any problems at all....
Sorry for the late response. Setting temp to 1 with the included presets seems to have fixed it!
I like the model. But I think I am starting to prefer the 32B now, with the 4090.
Do you ever plan to go beyond these sizes
@Nitral-AI
?
I see a lot of space for 32b finetunes. I'm having a hard time atm, may try to finetune in the future.
Sorry for the late response. Setting temp to 1 with the included presets seems to have fixed it!
I like the model. But I think I am starting to prefer the 32B now, with the 4090.
Do you ever plan to go beyond these sizes @Nitral-AI ?I see a lot of space for 32b finetunes. I'm having a hard time atm, may try to finetune in the future.
I'm bound to 12Gb's local for my own setup, so I tend not to bother training larger models. That and the elevated cost comes into play with something I essentially can't use myself effectively.
Sorry for the late response. Setting temp to 1 with the included presets seems to have fixed it!
I like the model. But I think I am starting to prefer the 32B now, with the 4090.
Do you ever plan to go beyond these sizes @Nitral-AI ?I see a lot of space for 32b finetunes. I'm having a hard time atm, may try to finetune in the future.
I'm bound to 12Gb's local for my own setup, so I tend not to bother training larger models. That and the elevated cost comes into play with something I essentially can't use myself effectively.
Understandable ^^