---
license: apache-2.0
---

<h1 align="center">mT5 small spanish es</h1>

This a Spanish fine-tuned model using as a starting point the base model mt5-small by Google.

        https://huggingface.co/google/mt5-small


# Datasets

The datasets used for the fine-tuning

        Task                                    Prefix
        Multinli (English)                      multi nli premise:[Text]  hypo:[Text]
        Multinli (Spanish)                      multi nli premise:[Text]  hypo:[Text]
        Pawx (English)                          pawx sentence1:[Text] sentence2:[Text]
        Pawx (Spanish)                          pawx sentence1:[Text] sentence2:[Text]
        Squad (English)                         question:[Text] context:[Text]
        Squad (Spanish)                         question:[Text] context:[Text]
        Translations (English-Spanish)          translate English to Spanish:[Text]
        Translations (Spanish-English)          translate Spanish to English:[Text]



# Inference

The following piece of code could be used to perfome the different model tasks.

Translations

        from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
        
        model_name = "HURIDOCS/mt5-small-spanish-es"
        
        tokenizer = AutoTokenizer.from_pretrained(model_name)
        model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
        
        task = "translate Spanish to English:Esta frase es para probar el modelo"
        input_ids = tokenizer(
            [task],
            return_tensors="pt",
            padding="max_length",
            truncation=True,
            max_length=512
        )["input_ids"]
        
        output_ids = model.generate(
            input_ids=input_ids,
            max_length=84,
            no_repeat_ngram_size=2,
            num_beams=4
        )[0]
        
        result_text = tokenizer.decode(
            output_ids,
            skip_special_tokens=True,
            clean_up_tokenization_spaces=False
        )
        
        print(result_text)


Question answering


        from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
        
        model_name = "HURIDOCS/mt5-small-spanish-es"
        
        tokenizer = AutoTokenizer.from_pretrained(model_name)
        model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
        
        task = '''question:En qué país se encuentra Normandía? context:Los normandos (normandos: Nourmann; Francés: Normandos; Normanni) 
        fue el pueblo que en los siglos X y XI dio su nombre a Normandía, una región de Francia. 
        Eran descendientes de invasores nórdicos ('normandos" viene de "Norseman") y piratas de Dinamarca, Islandia y Noruega que, 
        bajo su líder Rollo, acordaron jurar lealtad al rey Carlos III de Francia Occidental. A través de generaciones de asimilación 
        y mezcla con las poblaciones nativas francas y galas romanas, sus descendientes se fusionarían gradualmente con las culturas 
        carolingias de Francia Occidental. La identidad cultural y étnica distintiva de los normandos surgió inicialmente en la 
        primera mitad del siglo X, y continuó evolucionando durante los siglos siguientes.'''

        input_ids = tokenizer(
            [task],
            return_tensors="pt",
            padding="max_length",
            truncation=True,
            max_length=512
        )["input_ids"]
        
        output_ids = model.generate(
            input_ids=input_ids,
            max_length=84,
            no_repeat_ngram_size=2,
            num_beams=4
        )[0]
        
        result_text = tokenizer.decode(
            output_ids,
            skip_special_tokens=True,
            clean_up_tokenization_spaces=False
        )
        
        print(result_text)

# Fine-tuning

Check out the Transformers Libray examples

https://github.com/huggingface/transformers/tree/main/examples/pytorch/question-answering


# Performance

Spanish SQuAD v2 512 tokens

                  Model                                            Exact match     F1
        rank 1    mrm8488/distill-bert-base-spanish-wwm-cased      50.43%          71.45%
        rank 2    **mT5 small spanish es**                         48.35%          62.03%
        rank 3    flan-t5-small                                    41.44%          56.48%