---
language:
- en
metrics:
- accuracy
base_model:
- google-bert/bert-base-uncased
pipeline_tag: text-classification
library_name: transformers
---

# PLEASE CHECK ED FOR TOKEN






# Model Evaluation Guide

This document provides the necessary instructions to evaluate a pre-trained sequence classification model using a test dataset.

## Prerequisites

Before running the evaluation pipeline, ensure you have the following installed:

- Python 3.7+
- Required Python libraries  
  Install them by running:

```bash
pip install transformers datasets evaluate torch
```

## Dataset Information

The test dataset is hosted on the Hugging Face Hub under the namespace `CIS5190ml/Dataset`. The dataset should have the following structure:
- Column: `title`
- Column: `label`

Example entries:
- "Jack Carr's take on the late Tom Clancy..." (label: 0)
- "Feeding America CEO asks community to help..." (label: 0)
- "Trump's campaign rival decides between..." (label: 0)

## Model Information

The model being evaluated is hosted under the Hugging Face Hub namespace `CIS5190ml/bert4`.

## Evaluation Pipeline

The complete evaluation pipeline is provided in the file:
**Evaluation_Pipeline.ipynb**

This Jupyter Notebook walks you through the following steps:
1. Loading the pre-trained model and tokenizer
2. Loading and preprocessing the test dataset
3. Running predictions on the test data
4. Computing the evaluation metric (e.g., accuracy)

## Quick Start

Clone this repository and navigate to the directory:

```bash
git clone <repository-url>
cd <repository-directory>
```

Open the Jupyter Notebook:

```bash
jupyter notebook Evaluation_Pipeline.ipynb
```

Follow the step-by-step instructions in the notebook to evaluate the model.

## Code Example

Here is an overview of the evaluation pipeline used in the notebook:

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from datasets import load_dataset
import evaluate
import torch
from torch.utils.data import DataLoader

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("CIS5190ml/bert4")
model = AutoModelForSequenceClassification.from_pretrained("CIS5190ml/bert4")

# Load dataset
ds = load_dataset("CIS5190ml/test_20_rows", split="train")

# Preprocessing
def preprocess_function(examples):
    return tokenizer(examples["title"], truncation=True, padding="max_length")

encoded_ds = ds.map(preprocess_function, batched=True)
encoded_ds = encoded_ds.remove_columns([col for col in encoded_ds.column_names if col not in ["input_ids", "attention_mask", "label"]])
encoded_ds.set_format("torch")

# Create DataLoader
test_loader = DataLoader(encoded_ds, batch_size=8)

# Evaluate
accuracy = evaluate.load("accuracy")
model.eval()

for batch in test_loader:
    with torch.no_grad():
        outputs = model(input_ids=batch["input_ids"], attention_mask=batch["attention_mask"])
        preds = torch.argmax(outputs.logits, dim=-1)
        accuracy.add_batch(predictions=preds, references=batch["label"])

final_accuracy = accuracy.compute()
print("Accuracy:", final_accuracy["accuracy"])
```

## Output

After running the pipeline, the evaluation metric (e.g., accuracy) will be displayed in the notebook output. Example:

```
Accuracy: 0.85
```

## Notes

* If your dataset or column names differ, update the relevant sections in the notebook.
* To use a different evaluation metric, modify the `evaluate.load()` function in the notebook.
* For any issues or questions, please feel free to reach out.