aipracticecafe/calm2-7b-chat-wataoshi

Model Card for Model ID

This is a finetuned version of Calm2-7b-chat. The dataset used was the wikipedia article for 「わたおし」in english I'm in Love with the Villainess. The finetuning process wass to use LoRA, with the CausalLM task, simple next token prediction. It was done in the base model, then the LoRA weights were merged with the chat version of Calm2. The usage is the same as the Calm2-7b-chat from CyberAgent.

Base Model:

calm-2-7b-chat (https://huggingface.co/cyberagent/calm2-7b-chat)

Uses

The model is supossed to understand about the 「わたおし」 anime and should be able to answer questions about that work.

Training Details

Training Data

All the subpages from the website of the anime 「わたおし」, scrapped using trafilatura, and the wikipedia article for the anime. [More Information Needed]

Training Procedure

Using QLoRA with all linear layers as modules to finetune, rank and alpha both 64. Trained using Trainer from HuggingFace.

Citation [optional]

@article{touvron2023llama,
  title={LLaMA: Open and Efficient Foundation Language Models},
  author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
  journal={arXiv preprint arXiv:2302.13971},
  year={2023}
}

Model Card Authors [optional]

aipracticecafe