File size: 5,356 Bytes
68a005a b44bc11 68a005a b44bc11 d36fca7 68a005a b44bc11 68a005a fb0f0b6 86fcb7d a8e05da 86fcb7d fb0f0b6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 |
---
library_name: transformers
license: apache-2.0
datasets:
- NickyNicky/ngxson_MiniThinky_v1_deduplicated_11_percent
language:
- en
base_model:
- meta-llama/Llama-3.2-1B-Instruct
pipeline_tag: text-generation
---
## max tokens 512
## full full fine tune
## script base.
https://gist.github.com/willccbb/4676755236bb08cab5f4e54a0475d6fb
## Note.
```
I have taken the script from the user 'willccbb' and I have modified it many things to be able to train it.
thanks to this script I already understand many things.
```
## Semantic Comparison of Sentences and Mapping the Cosine Similarity to the [0 ... 0.123] Scale (rewards).
```
https://colab.research.google.com/drive/1hdWSImb9lJ7Q4Csh3ubLAblhD0EscDPh?usp=sharing
```
```py
import torch,gc
import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, BitsAndBytesConfig
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
HfArgumentParser,
TrainingArguments,
pipeline,
logging,
GenerationConfig,
TextIteratorStreamer,
)
from transformers import StoppingCriteria, StoppingCriteriaList
model_name= "NickyNicky/Llama-1B-base-GRPO-miniThinky_v1"
base_model = AutoModelForCausalLM.from_pretrained(model_name,
# quantization_config=bnb_config,
device_map={"":0},
# token=hf_token,
# attn_implementation="flash_attention_2",
torch_dtype=torch.bfloat16
)
max_seq_length=4048
base_tokenizer_0 = AutoTokenizer.from_pretrained(model_name,
# token=access_token,
max_length=max_seq_length)
prompt = "What do you mean by 'core' when referring to a subject or topic?"
sys="""Respond in the following format:
<reasoning>
...
</reasoning>
<answer>
...
</answer>
"""
messages = [
{"role": "system", "content": sys},
{"role": "user", "content": prompt}
]
device = "cuda" # the device to load the model onto
text = base_tokenizer_0.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = base_tokenizer_0([text], return_tensors="pt").to(device)
generated_ids = base_model.generate(
model_inputs.input_ids,
max_new_tokens=1200,
pad_token_id= base_tokenizer_0.eos_token_id,
)
generated_ids = [
output_ids[:] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = base_tokenizer_0.batch_decode(generated_ids, skip_special_tokens=True)[0]
messages.append({"role": "assistant", "content": response})
print(response)
'''
<|begin_of_text|><|begin_of_text|><|start_header_id|>system<|end_header_id|>
Cutting Knowledge Date: December 2023
Today Date: 31 Jan 2025
Respond in the following format:
<reasoning>
...
</reasoning>
<answer>
...
</answer><|eot_id|><|start_header_id|>user<|end_header_id|>
What do you mean by 'core' when referring to a subject or topic?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
<reasoning>
When referring to a "core" in a scientific context, it typically refers to the central, innermost part of the brain or central nervous system, which houses the brain's main processing centers, including the cerebrum, the hippocampus, and the amygdala. The core is involved in controlling and coordinating the body's functions, such as the regulation of the heart rate, blood pressure, and digestion, as well as the processing of sensory information from the environment.
In a more general sense, a core can also refer to the central part of the brain, which is the main processing center of the brain, often containing the brain's main neurons and is involved in the processing of complex information, including the integration of sensory data from the environment, the interpretation of emotions, and the generation of the body's response to the environment.
The core is also often used to describe the central nervous system's main processing centers, which are located in the brain and are responsible for controlling the body's functions, including the regulation of the body's internal and external environment, the processing of the senses, and the generation of the body's response to the environment.
</reasoning>
<answer>
The core is a central part of the brain that plays a vital role in controlling and coordinating the body's functions, and it is involved in the processing of complex information from the environment, the interpretation of emotions, and the generation of the body's response to the environment. It is a central processing center of the brain, and its functions are often compared to the functions of the heart, which is often referred to as the "heart of the brain." The core is also a key component of the brain's main processing centers, which are located in the cerebrum, the hippocampus, and the amygdala.
</answer>
<|eot_id|>
'''
```
|