YosefA commited on
Commit
17e4f80
Β·
verified Β·
1 Parent(s): c54ba7e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +109 -37
README.md CHANGED
@@ -3,61 +3,133 @@ library_name: transformers
3
  license: mit
4
  base_model: Davlan/afro-xlmr-base
5
  tags:
6
- - generated_from_trainer
 
 
 
 
 
 
7
  metrics:
8
  - f1
9
  model-index:
10
- - name: adfluence-intent-model
11
- results: []
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ---
13
 
14
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
- should probably proofread and complete it, then remove this comment. -->
16
 
17
- # adfluence-intent-model
18
 
19
- This model is a fine-tuned version of [Davlan/afro-xlmr-base](https://huggingface.co/Davlan/afro-xlmr-base) on an unknown dataset.
20
- It achieves the following results on the evaluation set:
21
- - Loss: 0.4281
22
- - F1: 0.8063
23
 
24
- ## Model description
25
 
26
- More information needed
27
 
28
- ## Intended uses & limitations
29
 
30
- More information needed
 
 
 
 
31
 
32
- ## Training and evaluation data
33
 
34
- More information needed
35
 
36
- ## Training procedure
37
 
38
- ### Training hyperparameters
 
39
 
40
- The following hyperparameters were used during training:
41
- - learning_rate: 2e-05
42
- - train_batch_size: 16
43
- - eval_batch_size: 16
44
- - seed: 42
45
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
46
- - lr_scheduler_type: linear
47
- - num_epochs: 3
 
48
 
49
- ### Training results
 
 
50
 
51
- | Training Loss | Epoch | Step | Validation Loss | F1 |
52
- |:-------------:|:-----:|:----:|:---------------:|:------:|
53
- | No log | 1.0 | 160 | 0.5001 | 0.7852 |
54
- | No log | 2.0 | 320 | 0.4316 | 0.8101 |
55
- | No log | 3.0 | 480 | 0.4281 | 0.8063 |
56
 
57
 
58
- ### Framework versions
 
 
 
59
 
60
- - Transformers 4.52.4
61
- - Pytorch 2.6.0+cu124
62
- - Datasets 3.6.0
63
- - Tokenizers 0.21.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  license: mit
4
  base_model: Davlan/afro-xlmr-base
5
  tags:
6
+ - amharic
7
+ - text-classification
8
+ - sentiment-analysis
9
+ - intent-classification
10
+ - code-switching
11
+ - ethiopia
12
+ - adfluence-ai
13
  metrics:
14
  - f1
15
  model-index:
16
+ - name: Adfluence-AI Purchase Intent Classifier
17
+ results:
18
+ - task:
19
+ type: text-classification
20
+ dataset:
21
+ name: YosefA/Adflufence-ad-comments
22
+ type: YosefA/Adflufence-ad-comments
23
+ config: default
24
+ split: test
25
+ metrics:
26
+ - name: F1 (Weighted)
27
+ type: f1
28
+ value: 0.8101
29
+ datasets:
30
+ - YosefA/Adflufence-ad-comments
31
+ language:
32
+ - am
33
+ pipeline_tag: text-classification
34
  ---
35
 
36
+ # Adfluence-AI Purchase Intent Classifier
 
37
 
38
+ This model is a fine-tuned version of **[Davlan/afro-xlmr-base](https://huggingface.co/Davlan/afro-xlmr-base)**, a powerful multilingual model with a strong understanding of African languages. It has been specifically trained to classify purchase intent in social media comments written in Amharic (Ge'ez script), Romanized Amharic, and mixed Amharic-English (code-switching).
39
 
40
+ This model was developed for the **Adfluence AI** project, which aims to evaluate the effectiveness of influencer marketing campaigns in Ethiopia.
 
 
 
41
 
42
+ It achieves a **weighted F1-score of 0.81** on the evaluation set.
43
 
44
+ ## Model Description
45
 
46
+ The model takes a social media comment as input and outputs a prediction across five categories of purchase intent:
47
 
48
+ * `highly_likely`
49
+ * `likely`
50
+ * `neutral`
51
+ * `unlikely`
52
+ * `highly_unlikely`
53
 
54
+ This allows for a nuanced understanding of audience reaction beyond simple positive/negative sentiment, directly measuring the potential for user conversion.
55
 
56
+ ## How to Use
57
 
58
+ You can use this model directly with the `pipeline` function from the `transformers` library.
59
 
60
+ ```python
61
+ from transformers import pipeline
62
 
63
+ # Load the model from the Hub
64
+ model_id = "your-username/adfluence-intent-model" #<-- Replace with your model ID
65
+ classifier = pipeline("text-classification", model=model_id)
66
+
67
+ # --- Example Usage ---
68
+
69
+ # Example 1: Amharic (Ge'ez Script) - Clear intent
70
+ comment_1 = "ዋው α‰ αŒ£αˆ አαˆͺፍ αŠα‹! የቡ αŠα‹ αˆ›αŒαŠ˜α‰΅ α‹¨αˆα‰½αˆˆα‹?"
71
+ # Translation: "Wow, this is great! Where can I find it?"
72
 
73
+ # Example 2: Mixed Amharic-English - Neutral/Questioning
74
+ comment_2 = "Hmm, interesting. Price-u endet new?"
75
+ # Translation: "Hmm, interesting. How is the price?"
76
 
77
+ # Example 3: Romanized Amharic - Negative
78
+ comment_3 = "Ene enja minim altemechegnim, quality yelelew neger new."
79
+ # Translation: "I don't know, I didn't like it at all, it's a thing with no quality."
 
 
80
 
81
 
82
+ results = classifier([comment_1, comment_2, comment_3])
83
+ for comment, result in zip([comment_1, comment_2, comment_3], results):
84
+ print(f"Comment: {comment}")
85
+ print(f"Prediction: {result['label']}, Score: {result['score']:.4f}\n")
86
 
87
+ # Expected Output:
88
+ # Comment: ዋው α‰ αŒ£αˆ አαˆͺፍ αŠα‹! የቡ αŠα‹ αˆ›αŒαŠ˜α‰΅ α‹¨αˆα‰½αˆˆα‹?
89
+ # Prediction: highly_likely, Score: 0.9851
90
+ #
91
+ # Comment: Hmm, interesting. Price-u endet new?
92
+ # Prediction: neutral, Score: 0.9214
93
+ #
94
+ # Comment: Ene enja minim altemechegnim, quality yelelew neger new.
95
+ # Prediction: highly_unlikely, Score: 0.9902
96
+
97
+ Intended Uses & Limitations
98
+ Intended Use
99
+ This model is intended to be used as a backend component for the Adfluence AI platform. Its primary purpose is to analyze user comments on social media advertisements (e.g., on Instagram, Facebook, TikTok) to gauge audience purchase intent and provide campaign performance metrics.
100
+ Limitations
101
+ Simulated Data: The model is trained on a high-quality simulated dataset, not on live social media data. While designed to reflect real-world usage, performance may vary on wild, un-sanitized data.
102
+ Domain Specificity: The source data was derived from product reviews (specifically for electronics). The model's performance may be strongest in the e-commerce/product domain and might require further fine-tuning for vastly different domains like services, events, or fashion.
103
+ Language Scope: The model only understands Amharic and English. It has not been trained on other Ethiopian languages like Tigrinya, Oromo, etc.
104
+ Training and Evaluation Data
105
+ This model was fine-tuned on the custom YosefA/Adflufence-ad-comments dataset.
106
+ The dataset was created through the following process:
107
+ Source: Started with ~5,000 English product reviews from an Amazon dataset.
108
+ Transformation: Each review was programmatically rephrased and translated into a simulated social media comment using Google's Gemini Flash.
109
+ Stylization: Comments were generated in three styles to mimic real-world Ethiopian user behavior:
110
+ Amharic (Ge’ez script)
111
+ Romanized Amharic
112
+ Mixed Amharic-English (Code-Switching)
113
+ Enrichment: Comments were styled with emojis, slang, and informal sentence structures.
114
+ Labeling: Each comment was assigned a purchase intent label mapped from the original star rating of the source review.
115
+ Training Procedure
116
+ Training Hyperparameters
117
+ The following hyperparameters were used during training:
118
+ learning_rate: 2e-05
119
+ train_batch_size: 16
120
+ eval_batch_size: 16
121
+ seed: 42
122
+ optimizer: AdamW with betas=(0.9,0.999) and epsilon=1e-08
123
+ lr_scheduler_type: linear
124
+ num_epochs: 3
125
+ Training Results
126
+ The model achieved its best performance at the end of Epoch 2.
127
+ Training Loss Epoch Step Validation Loss F1 (Weighted)
128
+ No log 1.0 160 0.5001 0.7852
129
+ No log 2.0 320 0.4316 0.8101
130
+ No log 3.0 480 0.4281 0.8063
131
+ Framework Versions
132
+ Transformers 4.41.2
133
+ Pytorch 2.3.0+cu121
134
+ Datasets 2.19.0
135
+ Tokenizers 0.19.1