add inference script

README.md

@@ -19,6 +19,42 @@ base_model: codellama/CodeLlama-13b-Instruct-hf
 
  🤖 Model Description: Panda-Coder is a state-of-the-art LLM, a fine-tuned model, specifically designed to generate code based on natural language instructions. It's the result of relentless innovation and meticulous fine-tuning, all to make coding easier and more accessible for everyone.
 
+## Inference 
+
+```python
+import torch
+import transformers
+from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments,BitsAndBytesConfig
+
+prompt = f"""### Instruction:
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+Write a Python quickstart script to get started with TensorFlow
+
+### Input:
+
+### Response:
+"""
+
+input_ids = tokenizer(prompt, return_tensors="pt", truncation=True).input_ids.cuda()
+outputs = base_model.generate(input_ids=input_ids, max_new_tokens=512, do_sample=True, top_p=0.9,temperature=0.1,repetition_penalty=1.1)
+
+print(f"Output:\n{tokenizer.batch_decode(outputs.detach().cpu().numpy(), skip_special_tokens=True)[0][len(prompt):]}")
+```
+
+Output
+
+```bash
+Output:
+import tensorflow as tf
+
+# Create a constant tensor
+hello_constant = tf.constant('Hello, World!')
+
+# Print the value of the constant
+print(hello_constant)
+```
+
  ## 🔗 Key Features:
 
  🌟 NLP-Based Coding: With Panda-Coder, you can transform your plain text instructions into functional code effortlessly. No need to grapple with syntax and semantics - it understands your language.