Update README.md

README.md

@@ -22,24 +22,33 @@ metrics:
 - exact_match
 library_name: transformers
 pipeline_tag: question-answering
+co2_eq_emissions: 90
 ---
 
 # QAmembert
 
 ## Model Description
 
-We present **QAmemBERT**, which is a [CamemBERT base](https://huggingface.co/camembert-base) fine-tuned for the Question-Answering task for the French language on five French Q&A datasets composed of contexts and questions with their answers inside the context (= SQuAD v1 format) but also contexts and questions with their answers not inside the context (= SQuAD v2 format).
-This represents a total of over **138 061 questions/answers pairs used to finetune this model and 3,188 to test it**.
+We present **QAmemBERT**, which is a [CamemBERT base](https://huggingface.co/camembert-base) fine-tuned for the Question-Answering task for the French language on four French Q&A datasets composed of contexts and questions with their answers inside the context (= SQuAD v1 format) but also contexts and questions with their answers not inside the context (= SQuAD v2 format).
+All these datasets were concatenated into a single dataset that we called [frenchQA](https://huggingface.co/datasets/CATIE-AQ/frenchQA).
+This represents a total of over **221,348 questions/answers pairs used to finetune this model and 6,376 to test it**.
 
 
 ## Datasets
 
 | Dataset     | Format      | Train split | Dev split   | Test split  |
 | ----------- | ----------- | ----------- | ----------- | ----------- |
-| [PIAFv1.2](https://www.data.gouv.fr/en/datasets/piaf-le-dataset-francophone-de-questions-reponses/)| SQuAD v1    | 9 225 Q & A  | X  | X  |         
-| [FQuADv1.0](https://fquad.illuin.tech/)| SQuAD v1    | 20 731 Q & A | 3 188 Q & A  (not used in training because it serves as a test dataset) | 2 189 Q & A (not used in our work because not freely available)|         
-| [lincoln/newsquadfr](https://huggingface.co/datasets/lincoln/newsquadfr) | SQuAD v1    | 1 650 Q & A  | 455 Q & A (not used in our work) | 415 Q & A (not used in our work) |         
-| [pragnakalp/squad_v2_french_translated](https://huggingface.co/datasets/pragnakalp/squad_v2_french_translated)| SQuAD v2    | 79 069 Q & A  | X  | X  |                
+| [piaf](https://www.data.gouv.fr/en/datasets/piaf-le-dataset-francophone-de-questions-reponses/)| SQuAD v1    | 9 224 Q & A  | X  | X  |  
+| piaf_v2| SQuAD v2    | 9 224 Q & A  | X  | X  |         
+| [fquad](https://fquad.illuin.tech/)| SQuAD v1    | 20 731 Q & A | 3 188 Q & A  (not used in training because it serves as a test dataset) | 2 189 Q & A (not used in our work because not freely available)|         
+| fquad_v2 | SQuAD v2    | 20 731 Q & A | 3 188 Q & A  (not used in training because it serves as a test dataset) | X |         
+| [lincoln/newsquadfr](https://huggingface.co/datasets/lincoln/newsquadfr) | SQuAD v1    | 1 650 Q & A  | 455 Q & A (not used in our work) | X |           
+| lincoln/newsquadfr_v2 | SQuAD v2    | 1 650 Q & A  | 455 Q & A (not used in our work) | X |         
+| [pragnakalp/squad_v2_french_translated](https://huggingface.co/datasets/pragnakalp/squad_v2_french_translated)| SQuAD v2    | 79 069 Q & A  | X  | X  |         
+| pragnakalp/squad_v2_french_translated_v2| SQuAD v2    | 79 069 Q & A  | X  | X  |
+
+All these datasets were concatenated into a single dataset that we called [frenchQA](https://huggingface.co/datasets/CATIE-AQ/frenchQA).
+
 
 ## Evaluation results
 
@@ -99,7 +108,7 @@ result = qa({
     'context': "Le français est une langue indo-européenne de la famille des langues romanes dont les locuteurs sont appelés francophones. Elle est parfois surnommée la langue de Molière.  Le français est parlé, en 2023, sur tous les continents par environ 321 millions de personnes : 235 millions l'emploient quotidiennement et 90 millions en sont des locuteurs natifs. En 2018, 80 millions d'élèves et étudiants s'instruisent en français dans le monde. Selon l'Organisation internationale de la francophonie (OIF), il pourrait y avoir 700 millions de francophones sur Terre en 2050."
 })
 
-if result['score'] < 0.1:
+if result['score'] < 0.01:
     print("La réponse n'est pas dans le contexte fourni.")
 else :
     print(result['answer'])
@@ -110,12 +119,10 @@ else :
 ```python
 # details
 result
-{
-  "score": 0.9703257083892822,
-  "start": 269,
-  "end": 281,
-  "answer": "235 millions"
-}
+{'score': 0.9945194721221924,
+ 'start': 269,
+ 'end': 281,
+ 'answer': '235 millions'}
 ```
 
 
@@ -131,7 +138,7 @@ result = qa({
 Construite en deux ans par Gustave Eiffel et ses collaborateurs pour l'Exposition universelle de Paris de 1889, célébrant le centenaire de la Révolution française, et initialement nommée « tour de 300 mètres », elle est devenue le symbole de la capitale française et un site touristique de premier plan : il s’agit du quatrième site culturel français payant le plus visité en 2016, avec 5,9 millions de visiteurs. Depuis son ouverture au public, elle a accueilli plus de 300 millions de visiteurs."
 })
 
-if result['score'] < 0.1:
+if result['score'] < 0.01:
     print("La réponse n'est pas dans le contexte fourni.")
 else :
     print(result['answer'])
@@ -142,14 +149,23 @@ La réponse n'est pas dans le contexte fourni.
 ```python
 # details
 result
-{
-  "score": 0.00011322159843984991,
-  "start": 0,
-  "end": 14,
-  "answer": "La tour Eiffel"
-}
+{'score': 3.619904940035945e-13,
+ 'start': 734,
+ 'end': 744,
+ 'answer': 'visiteurs.'}
 ```
 
+## Environmental Impact
+
+*Carbon emissions were estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). The hardware, runtime, cloud provider, and compute region were utilized to estimate the carbon impact.*
+
+- **Hardware Type:** A100 PCIe 40/80GB
+- **Hours used:** 4.96
+- **Cloud Provider:** Private Infrastructure
+- **Carbon Efficiency (kg/kWh):** 0.076kg (estimated from [electricitymaps](https://app.electricitymaps.com/zone/FR) ; we take the average carbon intensity in France for the month of March 2023, as we are unable to use the data for the day of training, which are not available.)
+- **Carbon Emitted** *(Power consumption x Time x Carbon produced based on location of power grid)*: 0.09 kg eq. CO2
+
+
 ## Citations
 
 ### PIAF