Upload README.md

README.md

@@ -0,0 +1,65 @@
+---
+license: cc-by-nc-4.0
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- DeepNight
+- deepnight-research
+- Orca
+- llama-2
+- llama-2-70b
+---
+
+
+# llama-2-70B-inst
+
+## Model Details
+
+* **Developed by**: [DeepNight](https://deepnight.tech)
+* **Backbone Model**: [LLaMA-2](https://github.com/facebookresearch/llama/tree/main)
+
+## Dataset Details
+
+### Used Datasets
+- ElutherAI/pile (30%)
+- TogetherComputer/Long-Data-Collections
+
+
+### Prompt Template
+```
+### System:
+{System}
+### User:
+{User}
+### Assistant:
+{Assistant}
+```
+
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
+tokenizer = AutoTokenizer.from_pretrained("deepnight-nexus/llama2-70b-inst")
+model = AutoModelForCausalLM.from_pretrained(
+    "deepnight-nexus/llama2-70b-inst",
+    device_map="auto",
+    torch_dtype=torch.float16,
+    load_in_8bit=True,
+    rope_scaling={"type": "dynamic", "factor": 2} # allows handling of longer inputs
+)
+prompt = "### User:\nThomas is healthy, but he has to go to the hospital. What could be the reasons?\n\n### Assistant:\n"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+del inputs["token_type_ids"]
+streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
+output = model.generate(**inputs, streamer=streamer, use_cache=True, max_new_tokens=float('inf'))
+output_text = tokenizer.decode(output[0], skip_special_tokens=True)
+```
+
+
+## Contact Us
+### About DeepNight
+- [DeepNight](https://deepnight.tech) is a company specialized in Large Language Models (LLMs) and AI. We will help you build private LLMs and related applications.
+If you have a dataset to build domain specific LLMs or make LLM applications, please contact us at ► [click here to contact](mailto:[email protected]?subject=Inquiry%20Regarding%20Custom%20LLM%20Development)