Spaces:
Running
Running
[ADD] Carvalho model
Browse files
app.py
CHANGED
@@ -5,7 +5,7 @@ import torch
|
|
5 |
from transformers import pipeline
|
6 |
|
7 |
# Model, information and examples ----------------------------------------------
|
8 |
-
MODEL_NAMES = ["Carballo-bloom-1.3B","Carballo-cerebras-1.3B"]
|
9 |
markdown_description_gl = """
|
10 |
[*English below*]
|
11 |
|
@@ -18,6 +18,10 @@ Este espazo contén diferentes Grandes Modelos da Linguaxe feitos para o galego
|
|
18 |
🧠 **[Carballo-cerebras-1.3B](https://huggingface.co/proxectonos/Carballo-cerebras-1.3B)** é un modelo de parámetros 1.3B baseado en [Cerebras-GPT 1.3B](https://huggingface.co/cerebras/Cerebras-GPT-1.3B).
|
19 |
|
20 |
👀 **Máis información sobre Carballo-cerebras-1.3B:** [tarxeta modelo oficial HF](https://huggingface.co/proxectonos/Carballo-cerebras-1.3B)
|
|
|
|
|
|
|
|
|
21 |
"""
|
22 |
|
23 |
markdown_description_en = """
|
@@ -31,6 +35,10 @@ This space contains the Galician language models developed by [Proxecto Nós](ht
|
|
31 |
🧠 **[Carballo-cerebras-1.3B](https://huggingface.co/proxectonos/Carballo-cerebras-1.3B)** is a 1.3B parameters model based in [Cerebras-GPT 1.3B](https://huggingface.co/cerebras/Cerebras-GPT-1.3B).
|
32 |
|
33 |
👀 **Learn more about Carballo-cerebras-1.3B:** [HF official model card](https://huggingface.co/proxectonos/Carballo-cerebras-1.3B)
|
|
|
|
|
|
|
|
|
34 |
"""
|
35 |
|
36 |
short_prompts_examples = [
|
@@ -47,17 +55,21 @@ few_shot_prompts_examples = [
|
|
47 |
fronted_theme = 'Soft'
|
48 |
|
49 |
# Model charge ---------------------------------------------------------
|
50 |
-
|
51 |
-
|
52 |
model_id_cerebras = "proxectonos/Carballo-cerebras-1.3B"
|
53 |
generator_model_cerebras = pipeline("text-generation", model=model_id_cerebras)
|
|
|
|
|
54 |
|
55 |
# Generation functions ---------------------------------------------------------
|
56 |
def get_model(model_selection):
|
57 |
if model_selection == "Carballo-bloom-1.3B":
|
58 |
-
return
|
59 |
-
|
60 |
return generator_model_cerebras
|
|
|
|
|
61 |
|
62 |
def remove_empty_lines(text):
|
63 |
lines = text.strip().split("\n")
|
|
|
5 |
from transformers import pipeline
|
6 |
|
7 |
# Model, information and examples ----------------------------------------------
|
8 |
+
MODEL_NAMES = ["Carballo-bloom-1.3B","Carballo-cerebras-1.3B","Carvalho_pt-gl"]
|
9 |
markdown_description_gl = """
|
10 |
[*English below*]
|
11 |
|
|
|
18 |
🧠 **[Carballo-cerebras-1.3B](https://huggingface.co/proxectonos/Carballo-cerebras-1.3B)** é un modelo de parámetros 1.3B baseado en [Cerebras-GPT 1.3B](https://huggingface.co/cerebras/Cerebras-GPT-1.3B).
|
19 |
|
20 |
👀 **Máis información sobre Carballo-cerebras-1.3B:** [tarxeta modelo oficial HF](https://huggingface.co/proxectonos/Carballo-cerebras-1.3B)
|
21 |
+
|
22 |
+
🧠 **[Carvalho_pt-gl](https://huggingface.co/Nos-PT/Carvalho_pt-gl-1.3B)** é un modelo de parámetros 1.3B baseado en [Cerebras-GPT 1.3B](https://huggingface.co/cerebras/Cerebras-GPT-1.3B) que pode traballar en galego e portugués, feito en colaboración co [VISTA Lab]() da Universidade de Évora.
|
23 |
+
|
24 |
+
👀 **Máis información sobre Carvalho_pt-gl:** [tarxeta modelo oficial HF](https://huggingface.co/Nos-PT/Carvalho_pt-gl-1.3B)
|
25 |
"""
|
26 |
|
27 |
markdown_description_en = """
|
|
|
35 |
🧠 **[Carballo-cerebras-1.3B](https://huggingface.co/proxectonos/Carballo-cerebras-1.3B)** is a 1.3B parameters model based in [Cerebras-GPT 1.3B](https://huggingface.co/cerebras/Cerebras-GPT-1.3B).
|
36 |
|
37 |
👀 **Learn more about Carballo-cerebras-1.3B:** [HF official model card](https://huggingface.co/proxectonos/Carballo-cerebras-1.3B)
|
38 |
+
|
39 |
+
📖 **[Carvalho_pt-gl](https://huggingface.co/Nos-PT/Carvalho_pt-gl-1.3B)** is a 1.3B parameters model based in [Cerebras-GPT 1.3B](https://huggingface.co/cerebras/Cerebras-GPT-1.3B) that can work in Galician and Portuguese, developed in collaboration with the [VISTA Lab](https://www.uevora.pt/en/research/R-D-Unit/vista-lab) at the University of Évora.
|
40 |
+
|
41 |
+
👀 **More information about Carvalho_pt-gl:** [official HF model card](https://huggingface.co/Nos-PT/Carvalho_pt-gl-1.3B)
|
42 |
"""
|
43 |
|
44 |
short_prompts_examples = [
|
|
|
55 |
fronted_theme = 'Soft'
|
56 |
|
57 |
# Model charge ---------------------------------------------------------
|
58 |
+
model_id_bloom = "proxectonos/Carballo-bloom-1.3B"
|
59 |
+
generator_model_bloom = pipeline("text-generation", model=model_id_bloom)
|
60 |
model_id_cerebras = "proxectonos/Carballo-cerebras-1.3B"
|
61 |
generator_model_cerebras = pipeline("text-generation", model=model_id_cerebras)
|
62 |
+
model_id_carvalho = "Nos-PT/Carvalho_pt-gl-1.3B"
|
63 |
+
generator_model_carvalho = pipeline("text-generation", model=model_id_carvalho)
|
64 |
|
65 |
# Generation functions ---------------------------------------------------------
|
66 |
def get_model(model_selection):
|
67 |
if model_selection == "Carballo-bloom-1.3B":
|
68 |
+
return generator_model_bloom
|
69 |
+
elif model_selection == "Carballo-cerebras-1.3B":
|
70 |
return generator_model_cerebras
|
71 |
+
else:
|
72 |
+
return generator_model_carvalho
|
73 |
|
74 |
def remove_empty_lines(text):
|
75 |
lines = text.strip().split("\n")
|