| # BERT-Base-Uncased Quantized Model for Twitter Tweet Sentiment Classification | |
| This repository hosts a quantized version of the **T5-Base** model, fine-tuned for **Movie Script Writting**. The model is optimized for efficient deployment while maintaining high accuracy, making it suitable for resource-constrained environments such as mobile and edge devices. | |
| ## Model Details | |
| - **Model Architecture:** T5-Base | |
| - **Task:** Movie Script Writting | |
| - **Dataset:** bookcorpus | |
| - **Quantization:** Float16 (FP16) | |
| - **Fine-tuning Framework:** Hugging Face Transformers | |
| - **Inference Framework:** PyTorch | |
| ## Usage | |
| ### Installation | |
| ```sh | |
| pip install transformers torch | |
| ``` | |
| ### Loading the Model | |
| ```python | |
| from transformers import BertForSequenceClassification, BertTokenizer | |
| import torch | |
| # Load quantized model | |
| quantized_model_path = "path/to/bert_finetuned_fp16" | |
| def generate_script(prompt): | |
| device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # Check available device | |
| model.to(device) # Move model to the appropriate device | |
| inputs = tokenizer(f"Generate a movie script: {prompt}", return_tensors="pt", truncation=True, padding="max_length", max_length=256) | |
| inputs = {key: value.to(device) for key, value in inputs.items()} # Move inputs to same device as model | |
| with torch.no_grad(): | |
| outputs = model.generate(**inputs, max_length=256, num_return_sequences=1) | |
| return tokenizer.decode(outputs[0], skip_special_tokens=True) | |
| # Test the script generator | |
| prompt = "SCENE: EXT. DARK ALLEY - NIGHT" | |
| print(generate_script(prompt)) | |
| ## Performance Metrics | |
| - **Accuracy:** 0.82 | |
| - **Inference Speed:** Faster due to FP16 quantization | |
| ## Fine-Tuning Details | |
| ### Dataset | |
| ### Training Configuration | |
| - **Number of epochs:** 3 | |
| - **Batch size:** 8 | |
| - **Evaluation strategy:** Per epoch | |
| - **Learning rate:** 2e-5 | |
| - **Optimizer:** AdamW | |
| ### Quantization | |
| The model is quantized using **Post-Training Quantization (PTQ)** with **Float16 (FP16)**, which reduces model size and improves inference efficiency while maintaining accuracy. | |
| ## Repository Structure | |
| ``` | |
| . | |
| ├── model/ # Contains the quantized model files | |
| ├── tokenizer_config/ # Tokenizer configuration and vocabulary files | |
| ├── model.safensors/ # Fine-tuned and quantized model | |
| ├── README.md # Model documentation | |
| ``` | |
| ## Limitations | |
| - The model is optimized for English-language next-word prediction tasks. | |
| - While quantization improves speed, minor accuracy degradation may occur. | |
| - Performance on out-of-distribution text (e.g., highly technical or domain-specific data) may be limited. | |
| ## Contributing | |
| Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements. | |
| `` | |