README.md

---
language:
- it
tags:
- twitter
- political-leaning
- politics
datasets:
- politic-it
widget:
- text: >-
    È necessario garantire salari dignitosi e condizioni di lavoro adeguate per
    tutelare i diritti dei lavoratori
  example_title: Left-wing example
- text: >-
    L'immigrazione deve essere gestita con rigore per preservare l'identità
    nazionale!
  example_title: Right-wing example
model-index:
- name: bert-political-leaning-it
  results:
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      type: social-media
      name: politic-it
    metrics:
    - type: f1 macro
      value: 61.3
    - type: accuracy
      value: 69.4
license: apache-2.0
metrics:
- f1
- accuracy
pipeline_tag: text-classification
---

# MattiaSangermano/bert-political-leaning-it

This model categorizes the political leaning of an Italian sentence into 4 categories: `moderate_left`, `left`, `right`, `moderate_right`. The model is a fine-tuned version of [neuraly/bert-base-italian-cased-sentiment](https://huggingface.co/neuraly/bert-base-italian-cased-sentiment). 

- **Developed by:** [Mattia Sangermano](https://www.linkedin.com/in/mattia-sangermano/) and [Fabio Murgese](https://www.linkedin.com/in/fabio-murgese/)
- **Model type:** Bert
- **Language(s) (NLP):** it
- **License:** Apache 2.0

### How to Get Started with the Model

You can use this model directly with a pipeline for text classification:

``` python
from transformers import pipeline
classifier = pipeline("text-classification",model='MattiaSangermano/bert-political-leaning-it')
prediction = classifier("Sovranità nazionale e identità forte")
print(prediction)
```

Here is how to use this model to classify a text in PyTorch:

``` python
from transformers import BertForSequenceClassification, AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained('MattiaSangermano/bert-political-leaning-it')
model = BertForSequenceClassification.from_pretrained('MattiaSangermano/bert-political-leaning-it')
tokens = tokenizer("Uguaglianza e giustizia sociale", return_tensors='pt')
logits = model(**tokens)[0]
prediction = model.config.id2label[torch.argmax(logits).item()]
print(prediction)
```

and in TensorFlow:

``` python
from transformers import AutoTokenizer, TFBertForSequenceClassification
import tensorflow as tf
tokenizer = AutoTokenizer.from_pretrained('MattiaSangermano/bert-political-leaning-it')
model = TFBertForSequenceClassification.from_pretrained('MattiaSangermano/bert-political-leaning-it')
tokens = tokenizer("Ambiente sano, futuro sicuro", padding=True,truncation=True,return_tensors='tf')
logits = model(tokens)[0]
prediction = model.config.id2label[tf.argmax(logits,1)[0].numpy()]
print(prediction)
```


### Out-of-Scope Use

It is important to recognize that political leaning is a personal and complex aspect of an individual's identity and attempting to classify it can be considered unethical and raise significant concerns. Therefore, the model should not be used to identify or classify the political orientation of individual users, nor should it be used for unethical purposes.

## Bias, Risks, and Limitations
During the construction of the dataset, deliberate efforts were made to exclude the names of politicians and political parties. As a result, these specific names might not hold relevance to the model.


## Dataset

We trained the model using the [PoliticIT](https://codalab.lisn.upsaclay.fr/competitions/8507#learn_the_details) competition dataset. The dataset was collected during 2020 and 2022 from the Twitter accounts of Italian politicians. These users were selected because their political affiliation can be guessed according to the party to which politicians belong to. The goal of the task was to classify a cluster of tweets, where a cluster is composed of texts written by different users that share the user self-assigned gender and the political ideology.

### Preprocessing

According to PoliticIT mantainers, from the dataset were discarded those tweets that contain mentions to news sites or some linguistic clues, such as the pipe symbol, which is used commonly by news sites to categorise their news. Moreover, the Twitter mentions were anonymised by replacing them with the token @user. Therefore the text traits cannot be guessed trivially by reading polititian's name and searching information on them on the Internet. Overall, the dataset consists of 103840 tweets.

#### Training Procedure 

The Dataset was split into train and validation sets with a stratified split having a ratio of 80-20. Although the main task of the original competition was to classify clusters of tweets this model was trained to predict only the political leaning of individual tweets. 

### Training Hyperparameters

- *Optimizer*: **Adam** with learning rate of **4e-5**, epsilon of **1e-7**
- *Loss*: **Categorical Cross Entropy** using **balanced** class weights
- *Max epochs*: **10**
- *Batch size*: **64**
- *Early Stopping*: monitoring validation loss with patience = **3**
- *Training regime*: fp16 mixed precision

## Evaluation
- test **f1-macro**: 61.3
- test **accuracy**: 69.4


| Avg Type | Precision | Recall | F1-score | Accuracy |
| ------ | ------ | ------ | ------ | ------ |
| Macro | 0.67 | 0.61 | 0.61 | - |
| Weighted | 0.74 | 0.69 | 0.77 | 0.69 |