Update README.md
Browse files
README.md
CHANGED
@@ -3,61 +3,133 @@ library_name: transformers
|
|
3 |
license: mit
|
4 |
base_model: Davlan/afro-xlmr-base
|
5 |
tags:
|
6 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
metrics:
|
8 |
- f1
|
9 |
model-index:
|
10 |
-
- name:
|
11 |
-
results:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
---
|
13 |
|
14 |
-
|
15 |
-
should probably proofread and complete it, then remove this comment. -->
|
16 |
|
17 |
-
|
18 |
|
19 |
-
This model
|
20 |
-
It achieves the following results on the evaluation set:
|
21 |
-
- Loss: 0.4281
|
22 |
-
- F1: 0.8063
|
23 |
|
24 |
-
|
25 |
|
26 |
-
|
27 |
|
28 |
-
|
29 |
|
30 |
-
|
|
|
|
|
|
|
|
|
31 |
|
32 |
-
|
33 |
|
34 |
-
|
35 |
|
36 |
-
|
37 |
|
38 |
-
|
|
|
39 |
|
40 |
-
|
41 |
-
-
|
42 |
-
-
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
-
|
|
|
48 |
|
49 |
-
|
|
|
|
|
50 |
|
51 |
-
|
52 |
-
|
53 |
-
|
54 |
-
| No log | 2.0 | 320 | 0.4316 | 0.8101 |
|
55 |
-
| No log | 3.0 | 480 | 0.4281 | 0.8063 |
|
56 |
|
57 |
|
58 |
-
|
|
|
|
|
|
|
59 |
|
60 |
-
|
61 |
-
|
62 |
-
|
63 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
license: mit
|
4 |
base_model: Davlan/afro-xlmr-base
|
5 |
tags:
|
6 |
+
- amharic
|
7 |
+
- text-classification
|
8 |
+
- sentiment-analysis
|
9 |
+
- intent-classification
|
10 |
+
- code-switching
|
11 |
+
- ethiopia
|
12 |
+
- adfluence-ai
|
13 |
metrics:
|
14 |
- f1
|
15 |
model-index:
|
16 |
+
- name: Adfluence-AI Purchase Intent Classifier
|
17 |
+
results:
|
18 |
+
- task:
|
19 |
+
type: text-classification
|
20 |
+
dataset:
|
21 |
+
name: YosefA/Adflufence-ad-comments
|
22 |
+
type: YosefA/Adflufence-ad-comments
|
23 |
+
config: default
|
24 |
+
split: test
|
25 |
+
metrics:
|
26 |
+
- name: F1 (Weighted)
|
27 |
+
type: f1
|
28 |
+
value: 0.8101
|
29 |
+
datasets:
|
30 |
+
- YosefA/Adflufence-ad-comments
|
31 |
+
language:
|
32 |
+
- am
|
33 |
+
pipeline_tag: text-classification
|
34 |
---
|
35 |
|
36 |
+
# Adfluence-AI Purchase Intent Classifier
|
|
|
37 |
|
38 |
+
This model is a fine-tuned version of **[Davlan/afro-xlmr-base](https://huggingface.co/Davlan/afro-xlmr-base)**, a powerful multilingual model with a strong understanding of African languages. It has been specifically trained to classify purchase intent in social media comments written in Amharic (Ge'ez script), Romanized Amharic, and mixed Amharic-English (code-switching).
|
39 |
|
40 |
+
This model was developed for the **Adfluence AI** project, which aims to evaluate the effectiveness of influencer marketing campaigns in Ethiopia.
|
|
|
|
|
|
|
41 |
|
42 |
+
It achieves a **weighted F1-score of 0.81** on the evaluation set.
|
43 |
|
44 |
+
## Model Description
|
45 |
|
46 |
+
The model takes a social media comment as input and outputs a prediction across five categories of purchase intent:
|
47 |
|
48 |
+
* `highly_likely`
|
49 |
+
* `likely`
|
50 |
+
* `neutral`
|
51 |
+
* `unlikely`
|
52 |
+
* `highly_unlikely`
|
53 |
|
54 |
+
This allows for a nuanced understanding of audience reaction beyond simple positive/negative sentiment, directly measuring the potential for user conversion.
|
55 |
|
56 |
+
## How to Use
|
57 |
|
58 |
+
You can use this model directly with the `pipeline` function from the `transformers` library.
|
59 |
|
60 |
+
```python
|
61 |
+
from transformers import pipeline
|
62 |
|
63 |
+
# Load the model from the Hub
|
64 |
+
model_id = "your-username/adfluence-intent-model" #<-- Replace with your model ID
|
65 |
+
classifier = pipeline("text-classification", model=model_id)
|
66 |
+
|
67 |
+
# --- Example Usage ---
|
68 |
+
|
69 |
+
# Example 1: Amharic (Ge'ez Script) - Clear intent
|
70 |
+
comment_1 = "αα α α£α α αͺα αα! α¨α΅ αα αααα΅ α¨αα½αα?"
|
71 |
+
# Translation: "Wow, this is great! Where can I find it?"
|
72 |
|
73 |
+
# Example 2: Mixed Amharic-English - Neutral/Questioning
|
74 |
+
comment_2 = "Hmm, interesting. Price-u endet new?"
|
75 |
+
# Translation: "Hmm, interesting. How is the price?"
|
76 |
|
77 |
+
# Example 3: Romanized Amharic - Negative
|
78 |
+
comment_3 = "Ene enja minim altemechegnim, quality yelelew neger new."
|
79 |
+
# Translation: "I don't know, I didn't like it at all, it's a thing with no quality."
|
|
|
|
|
80 |
|
81 |
|
82 |
+
results = classifier([comment_1, comment_2, comment_3])
|
83 |
+
for comment, result in zip([comment_1, comment_2, comment_3], results):
|
84 |
+
print(f"Comment: {comment}")
|
85 |
+
print(f"Prediction: {result['label']}, Score: {result['score']:.4f}\n")
|
86 |
|
87 |
+
# Expected Output:
|
88 |
+
# Comment: αα α α£α α αͺα αα! α¨α΅ αα αααα΅ α¨αα½αα?
|
89 |
+
# Prediction: highly_likely, Score: 0.9851
|
90 |
+
#
|
91 |
+
# Comment: Hmm, interesting. Price-u endet new?
|
92 |
+
# Prediction: neutral, Score: 0.9214
|
93 |
+
#
|
94 |
+
# Comment: Ene enja minim altemechegnim, quality yelelew neger new.
|
95 |
+
# Prediction: highly_unlikely, Score: 0.9902
|
96 |
+
|
97 |
+
Intended Uses & Limitations
|
98 |
+
Intended Use
|
99 |
+
This model is intended to be used as a backend component for the Adfluence AI platform. Its primary purpose is to analyze user comments on social media advertisements (e.g., on Instagram, Facebook, TikTok) to gauge audience purchase intent and provide campaign performance metrics.
|
100 |
+
Limitations
|
101 |
+
Simulated Data: The model is trained on a high-quality simulated dataset, not on live social media data. While designed to reflect real-world usage, performance may vary on wild, un-sanitized data.
|
102 |
+
Domain Specificity: The source data was derived from product reviews (specifically for electronics). The model's performance may be strongest in the e-commerce/product domain and might require further fine-tuning for vastly different domains like services, events, or fashion.
|
103 |
+
Language Scope: The model only understands Amharic and English. It has not been trained on other Ethiopian languages like Tigrinya, Oromo, etc.
|
104 |
+
Training and Evaluation Data
|
105 |
+
This model was fine-tuned on the custom YosefA/Adflufence-ad-comments dataset.
|
106 |
+
The dataset was created through the following process:
|
107 |
+
Source: Started with ~5,000 English product reviews from an Amazon dataset.
|
108 |
+
Transformation: Each review was programmatically rephrased and translated into a simulated social media comment using Google's Gemini Flash.
|
109 |
+
Stylization: Comments were generated in three styles to mimic real-world Ethiopian user behavior:
|
110 |
+
Amharic (Geβez script)
|
111 |
+
Romanized Amharic
|
112 |
+
Mixed Amharic-English (Code-Switching)
|
113 |
+
Enrichment: Comments were styled with emojis, slang, and informal sentence structures.
|
114 |
+
Labeling: Each comment was assigned a purchase intent label mapped from the original star rating of the source review.
|
115 |
+
Training Procedure
|
116 |
+
Training Hyperparameters
|
117 |
+
The following hyperparameters were used during training:
|
118 |
+
learning_rate: 2e-05
|
119 |
+
train_batch_size: 16
|
120 |
+
eval_batch_size: 16
|
121 |
+
seed: 42
|
122 |
+
optimizer: AdamW with betas=(0.9,0.999) and epsilon=1e-08
|
123 |
+
lr_scheduler_type: linear
|
124 |
+
num_epochs: 3
|
125 |
+
Training Results
|
126 |
+
The model achieved its best performance at the end of Epoch 2.
|
127 |
+
Training Loss Epoch Step Validation Loss F1 (Weighted)
|
128 |
+
No log 1.0 160 0.5001 0.7852
|
129 |
+
No log 2.0 320 0.4316 0.8101
|
130 |
+
No log 3.0 480 0.4281 0.8063
|
131 |
+
Framework Versions
|
132 |
+
Transformers 4.41.2
|
133 |
+
Pytorch 2.3.0+cu121
|
134 |
+
Datasets 2.19.0
|
135 |
+
Tokenizers 0.19.1
|