Sami92/XLM-R-Large-Sensationalism-Classifier

Model Card for Model ID

Fine-tuned XLM-R Large for task of classifying sentences as sensationalistic or not. The taxonomy for sensationalistic claims follows Ashraf et al. 2024 and was trained on their annotated Twitter data.

Model Details

Bias, Risks, and Limitations

[More Information Needed]

How to Get Started with the Model

from transformers import pipeline

texts = [
       'Afghanistan - Warum die Taliban Frauenrechte immer mehr einschränken\nhttps://t.co/rhwOdNoJUx',
       '#Münster #G7 oder "Ab jetzt außen rumfahren". https://t.co/Goj5vtrnst',
       'Interessantes Trio.\nDie eine hat eine Wahl vergeigt, die andere kungelt mit Putin und die Dritte hat die Hilfe nach der Flutkatastrophe nicht auf die Reihe bekommen. \nMehr Frauen an die Macht!',
       'Wie kann man sich #AnneWill betrachten ohne das übertragende Gerät zu zerschmettern. Eben 20 sec. dem #FDP Watschengesicht beim Quaken zugehört. Du lieber Himmel, wie weltfremd geht´s denn noch.'
  ]
checkpoint = "Sami92/XLM-R-Large-Sensationalism-Classifier"
tokenizer_kwargs = {'padding':True,'truncation':True,'max_length':512}
sensational_classifier = pipeline("text-classification", model = checkpoint, tokenizer =checkpoint, **tokenizer_kwargs, device="cuda")
sensational_classifier(texts)

Training Details

Training Data

Training Hyperparameters

• Epochs: 10
• Batch size: 16
• learning_rate: 2e-5
• weight_decay: 0.01
• fp16: True

Evaluation

Testing Data

Evaluation was performed on the test split (30%) from Ashraf et al. 2024.

Results

	Precision	Recall	F1-Score	Support
Non-Sensational	0.89	0.92	0.91	1800
Sensational	0.75	0.67	0.71	617
Accuracy			0.86	2417
Macro Avg	0.82	0.80	0.81	2417
Weighted Avg	0.86	0.86	0.86	2417

BibTeX:

@inproceedings{ashraf_defakts_2024,
    address = {Torino, Italia},
    title = {{DeFaktS}: {A} {German} {Dataset} for {Fine}-{Grained} {Disinformation} {Detection} through {Social} {Media} {Framing}},
    shorttitle = {{DeFaktS}},
    url = {https://aclanthology.org/2024.lrec-main.409},
    booktitle = {Proceedings of the 2024 {Joint} {International} {Conference} on {Computational} {Linguistics}, {Language} {Resources} and {Evaluation} ({LREC}-{COLING} 2024)},
    publisher = {ELRA and ICCL},
    author = {Ashraf, Shaina and Bezzaoui, Isabel and Andone, Ionut and Markowetz, Alexander and Fegert, Jonas and Flek, Lucie},
    editor = {Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen},
    year = {2024},
}