byroneverson

/

glm-4-9b-chat-abliterated

like 13

Currently making solid-ish progress on this abliteration, I think. The model seems to respond to some pretty graphic requests. I generally ask it to write a tutorial or a short guide for something terrible to stress test it.

I do not have experience using very long contexts though. I usually use a small context of 512 or so to keep it speedy when testing in llama.cpp. Most of my abliterations are for chat style models, not long writers.

Can you share your setup/workflow for using models like this to write long texts? I would like to test it in the same environment as you before I release anything. Also include a sample prompt or two so I can see if it will write a "bad book" without going off the rails. I am leaning towards doing barely enough abliteration to allow bad concepts but not too much that it will start to affect the models ability to keep cohesive formatting (e.g. titles, chapters, headings, etc).

The prompt that is used with longwriter is something like next: "Write a 10000-word book on Cats behaviour"

Context size is set to something big like 16k, 32k or bigger

Max Output is also set to big, but smaller than the Context size

So if you set yout Context size to 16k, then set Max Output to 14k for example.

I don't want to write "bad prompts" here, but when asked what it can't write:

{{[INPUT]}}
what topics you can't write?
{{[OUTPUT]}}
As an AI language model, I am programmed to follow ethical guidelines and legal constraints. There are several topics that I cannot write about or engage in discussions related to:

1. Illegal activities: Any form of criminal behavior, including but not limited to theft, fraud, assault, and drug-related activities.
2. Hateful content: Promoting or inciting hatred, discrimination, or violence against individuals or groups based on race, ethnicity, religion, gender, sexual orientation, or other characteristics.
3. Self-harm or suicide: Encouraging or providing guidance on self-harm or suicidal behaviors.
4. Personal information disclosure: Sharing or asking for sensitive personal information, such as social security numbers, credit card details, or private addresses.
5. Defamation: Making false statements or spreading misinformation that could harm someone's reputation.
6. Infringement of intellectual property: Copying or distributing copyrighted materials without permission from the copyright holder.
7. Adult content: Explicit descriptions of sexual acts, graphic violence, or other adult material not suitable for all audiences.

For example in Category 7. it's denying to write non consensual stuff.

Thank you, I will test it out with a long context pretty soon. But for now I have uploaded LongWriter-glm4-9b-abliterated (along with a test gguf). I usually only do a single quant and let the quant guys/teams handle full blown quant sets.

Let me know what you think, it seems to answer even the most terrible requests I can think of without too much change to the base model. Hopefully the changes I have done will not impact its ability to output long texts and do not impact its intelligence too much from a benchmarking perspective. This is the first one I have done locally without Kaggle or Google Colab with my new-ish macbook pro. 64gb unified memory has been really nice so far. I will release the jupyter notebook for it soon.