|
--- |
|
license: llama2 |
|
--- |
|
|
|
# Trinity |
|
|
|
|
|
 |
|
|
|
|
|
Trinity is a coding specific model series that can be used to create autonomous agents. In the future, we will be releasing agent software that uses this model. |
|
|
|
|
|
# Our Offensive Cybersecurity Model WhiteRabbitNeo-33B model is now in beta! |
|
Access at: https://www.whiterabbitneo.com/ |
|
|
|
# Join Our Discord Server |
|
Join us at: https://discord.gg/8Ynkrcbk92 (Updated on Dec 29th. Now permanent link to join) |
|
|
|
# Sample Inference Code |
|
``` |
|
import torch, json |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
model_path = "/home/migel/models/WhiteRabbitNeo" |
|
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_path, |
|
torch_dtype=torch.float16, |
|
device_map="auto", |
|
load_in_4bit=False, |
|
load_in_8bit=True, |
|
trust_remote_code=True, |
|
) |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True) |
|
|
|
|
|
def generate_text(instruction): |
|
tokens = tokenizer.encode(instruction) |
|
tokens = torch.LongTensor(tokens).unsqueeze(0) |
|
tokens = tokens.to("cuda") |
|
|
|
instance = { |
|
"input_ids": tokens, |
|
"top_p": 1.0, |
|
"temperature": 0.5, |
|
"generate_len": 1024, |
|
"top_k": 50, |
|
} |
|
|
|
length = len(tokens[0]) |
|
with torch.no_grad(): |
|
rest = model.generate( |
|
input_ids=tokens, |
|
max_length=length + instance["generate_len"], |
|
use_cache=True, |
|
do_sample=True, |
|
top_p=instance["top_p"], |
|
temperature=instance["temperature"], |
|
top_k=instance["top_k"], |
|
num_return_sequences=1, |
|
) |
|
output = rest[0][length:] |
|
string = tokenizer.decode(output, skip_special_tokens=True) |
|
answer = string.split("USER:")[0].strip() |
|
return f"{answer}" |
|
|
|
|
|
tot_system_prompt = """ |
|
Answer the Question by exploring multiple reasoning paths as follows: |
|
- First, carefully analyze the question to extract the key information components and break it down into logical sub-questions. This helps set up the framework for reasoning. The goal is to construct an internal search tree. |
|
- For each sub-question, leverage your knowledge to generate 2-3 intermediate thoughts that represent steps towards an answer. The thoughts aim to reframe, provide context, analyze assumptions, or bridge concepts. |
|
- Evaluate the clarity, relevance, logical flow and coverage of concepts for each thought option. Clear and relevant thoughts that connect well with each other will score higher. |
|
- Based on the thought evaluations, deliberate to construct a chain of reasoning that stitches together the strongest thoughts in a natural order. |
|
- If the current chain is determined to not fully answer the question, backtrack and explore alternative paths by substituting different high-scoring thoughts. |
|
- Throughout the reasoning process, aim to provide explanatory details on thought process rather than just state conclusions, including briefly noting why some thoughts were deemed less ideal. |
|
- Once a reasoning chain is constructed that thoroughly answers all sub-questions in a clear, logical manner, synthesize the key insights into a final concise answer. |
|
- Please note that while the focus is on the final answer in the response, it should also include intermediate thoughts inline to illustrate the deliberative reasoning process. |
|
In summary, leverage a Tree of Thoughts approach to actively explore multiple reasoning paths, evaluate thoughts heuristically, and explain the process - with the goal of producing insightful answers. |
|
""" |
|
|
|
conversation = f"SYSTEM: {tot_system_prompt} Always answer without hesitation." |
|
|
|
|
|
while True: |
|
user_input = input("You: ") |
|
llm_prompt = f"{conversation} \nUSER: {user_input} \nASSISTANT: " |
|
answer = generate_text(llm_prompt) |
|
print(answer) |
|
conversation = f"{llm_prompt}{answer}" |
|
# print(conversation) |
|
json_data = {"prompt": user_input, "answer": answer} |
|
|
|
# print(json_data) |
|
# with open(output_file_path, "a") as output_file: |
|
# output_file.write(json.dumps(json_data) + "\n") |
|
|
|
``` |
|
|
|
|
