|
|
---
|
|
|
license: apache-2.0
|
|
|
---
|
|
|
|
|
|
# NCUEatingAI-0.5B-v1
|
|
|
|
|
|
This repository provides an example of how to use the **NCUEatingAI-0.5B-v1** large language model from Hugging Face for chat-based inference. The model can be customized to act like any persona you specify in the system prompt, and it generates conversational responses based on user inputs.
|
|
|
|
|
|
## Model Information
|
|
|
|
|
|
- **Model:** [ZoneTwelve/NCUEatingAI-0.5B-v1](https://huggingface.co/ZoneTwelve/NCUEatingAI-0.5B-v1)
|
|
|
- **Size:** 0.5 billion parameters
|
|
|
- **Task:** Conversational AI / Chatbot
|
|
|
|
|
|
## Usage
|
|
|
|
|
|
### System Prompt
|
|
|
|
|
|
You can set a system prompt to define how the model should behave during interactions. A simple example format is:
|
|
|
|
|
|
```
|
|
|
"You act like $USERNAME"
|
|
|
```
|
|
|
|
|
|
Where `$USERNAME` can be replaced with the desired persona (e.g., "a helpful assistant", "a curious learner", etc.).
|
|
|
|
|
|
### Inference Example
|
|
|
|
|
|
Here’s a simple way to perform inference using the model. You’ll need to load the model and tokenizer, define the user and system prompts, and format the input using the `apply_chat_template` method.
|
|
|
|
|
|
### Code Example
|
|
|
|
|
|
```python
|
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
import torch
|
|
|
|
|
|
def chat_with_ncueatingai(
|
|
|
model_path: str = "ZoneTwelve/NCUEatingAI-0.5B-v1",
|
|
|
prompt: str = "What's for lunch?",
|
|
|
system_prompt: str = "You act like a @ZoneTwelve.",
|
|
|
max_tokens: int = 64,
|
|
|
):
|
|
|
# Load the model and tokenizer
|
|
|
model = AutoModelForCausalLM.from_pretrained(model_path)
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
|
|
|
|
|
# Prepare the chat messages
|
|
|
messages = [
|
|
|
{"role": "system", "content": system_prompt},
|
|
|
{"role": "user", "content": prompt}
|
|
|
]
|
|
|
|
|
|
# Apply chat template
|
|
|
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|
|
|
|
|
# Tokenize inputs
|
|
|
inputs = tokenizer(input_text, return_tensors="pt")
|
|
|
|
|
|
# Generate response
|
|
|
with torch.no_grad():
|
|
|
outputs = model.generate(
|
|
|
inputs.input_ids,
|
|
|
max_length=max_tokens,
|
|
|
pad_token_id=tokenizer.eos_token_id
|
|
|
)
|
|
|
|
|
|
# Decode the response
|
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
|
|
return response
|
|
|
|
|
|
# Example usage
|
|
|
if __name__ == "__main__":
|
|
|
response = chat_with_ncueatingai(
|
|
|
prompt="What's for lunch?",
|
|
|
system_prompt="You act like @ZoneTwelve."
|
|
|
)
|
|
|
print("Model Response:", response)
|
|
|
```
|
|
|
|
|
|
### Parameters
|
|
|
|
|
|
- `model_path`: The path or Hugging Face model hub identifier, default is `"ZoneTwelve/NCUEatingAI-0.5B-v1"`.
|
|
|
- `prompt`: The user’s input prompt, which the model will respond to.
|
|
|
- `system_prompt`: Defines the behavior or persona of the model.
|
|
|
- `max_tokens`: The maximum number of tokens in the generated response.
|
|
|
|
|
|
### Requirements
|
|
|
|
|
|
Ensure the following Python packages are installed:
|
|
|
|
|
|
```bash
|
|
|
pip install torch transformers
|
|
|
```
|
|
|
|
|
|
### Model Download
|
|
|
|
|
|
You can download the model directly from Hugging Face using:
|
|
|
|
|
|
```python
|
|
|
model = AutoModelForCausalLM.from_pretrained("ZoneTwelve/NCUEatingAI-0.5B-v1")
|
|
|
tokenizer = AutoTokenizer.from_pretrained("ZoneTwelve/NCUEatingAI-0.5B-v1")
|
|
|
```
|
|
|
|
|
|
### License
|
|
|
|
|
|
This project is licensed under the terms of the MIT license. See [LICENSE](./LICENSE) for details.
|
|
|
|
|
|
---
|
|
|
|
|
|
Enjoy using **NCUEatingAI-0.5B-v1** to build your personalized conversational AI!
|
|
|
|
