Want to contribute? TheBloke's Patreon page

Kimiko 13B - FP16

Model creator: nRuaif
Original model: Kimiko 13B

Description

This repo contains pytorch format fp16 model files for none.

It is the result of merging and/or converting the source repository to float16.

Repositories available

Prompt template: %%PROMPT_TEMPLATE_TITLE

<<HUMAN>>
{prompt}

<<AIBOT>>

Discord

For further support, and discussions on these models and AI in general, join us at:

TheBloke AI's Discord server

Thanks, and how to contribute.

Thanks to the chirper.ai team!

I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.

If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.

Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.

Patreon: https://patreon.com/TheBlokeAI
Ko-Fi: https://ko-fi.com/TheBlokeAI

Special thanks to: Luke from CarbonQuill, Aemon Algiz.

Patreon special mentions: Slarti, Chadd, John Detwiler, Pieter, zynix, K, Mano Prime, ReadyPlayerEmma, Ai Maven, Leonard Tan, Edmond Seymore, Joseph William Delisle, Luke @flexchar, Fred von Graf, Viktor Bowallius, Rishabh Srivastava, Nikolai Manek, Matthew Berman, Johann-Peter Hartmann, ya boyyy, Greatston Gnanesh, Femi Adebogun, Talal Aujan, Jonathan Leane, terasurfer, David Flickinger, William Sang, Ajan Kanaga, Vadim, Artur Olbinski, Raven Klaugh, Michael Levine, Oscar Rangel, Randy H, Cory Kujawski, RoA, Dave, Alex, Alexandros Triantafyllidis, Fen Risland, Eugene Pentland, vamX, Elle, Nathan LeClaire, Khalefa Al-Ahmad, Rainer Wilmers, subjectnull, Junyu Yang, Daniel P. Andersen, SuperWojo, LangChain4j, Mandus, Kalila, Illia Dulskyi, Trenton Dambrowitz, Asp the Wyvern, Derek Yates, Jeffrey Morgan, Deep Realms, Imad Khwaja, Pyrater, Preetika Verma, biorpg, Gabriel Tamborski, Stephen Murray, Spiking Neurons AB, Iucharbius, Chris Smitley, Willem Michiel, Luke Pendergrass, Sebastain Graf, senxiiz, Will Dee, Space Cruiser, Karl Bernard, Clay Pascal, Lone Striker, transmissions 11, webtim, WelcomeToTheClub, Sam, theTransient, Pierre Kircher, chris gileta, John Villwock, Sean Connelly, Willian Hasse

Thank you to all my generous patrons and donaters!

Original model card: none

Model Card for Kimiko_13B

This is my new Kimiko models, trained with LLaMA2-13B for...purpose

Model Details

Model Description

Developed by: nRuaif
Model type: Decoder only
License: CC BY-NC-SA
Finetuned from model [optional]: LLaMA 2

Model Sources [optional]

Repository: https://github.com/OpenAccess-AI-Collective/axolotl

Uses

Direct Use

This model is trained on 3k examples of instructions dataset, high quality roleplay, for best result follow this format

<<HUMAN>>
How to do abc

<<AIBOT>>
Here is how

Or with system prompting for roleplay

<<SYSTEM>>
A's Persona:
B's Persona:
Scenario:
Add some instruction here on how you want your RP to go.

Bias, Risks, and Limitations

All bias of this model come from LlaMA2 with an exception of NSFW bias.....

Training Details

Training Data

3000 examples from LIMAERP, LIMA and I sample 1000 good instruction from Airboro

Training Procedure

Model is trained with 1 L4 from GCP costing a whooping 2.5USD

Training Hyperparameters

Training regime: [More Information Needed]

3 epochs with 0.0002 lr, full 4096 ctx token, QLoRA

Speeds, Sizes, Times [optional]

It takes 18 hours to train this model with xformers enable

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: L4 with 12CPUs 48gb ram
Hours used: 5
Cloud Provider: GCP
Compute Region: US
Carbon Emitted: 0.5KG

TheBloke
/

Kimiko-13B-fp16