Update README.md

README.md

@@ -5,13 +5,12 @@ datasets:
 license: cc-by-4.0
 ---
 
-# tinyroberta for Extractive QA
+# roberta-base distilled into tinyroberta
 
 ## Overview
-**Language model:** tinyroberta-squad2  
+**Language model:** roberta-base  
 **Language:** English  
 **Training data:** The PILE  
-**Code:**  See [an example extractive QA pipeline built with Haystack](https://haystack.deepset.ai/tutorials/34_extractive_qa_pipeline)  
 **Infrastructure**: 4x Tesla v100
 
 ## Hyperparameters
@@ -19,7 +18,6 @@ license: cc-by-4.0
 ```
 batch_size = 96
 n_epochs = 4
-base_LM_model = "deepset/tinyroberta-squad2-step1"
 max_seq_len = 384
 learning_rate = 1e-4
 lr_schedule = LinearWarmup
@@ -32,50 +30,6 @@ This model was distilled using the TinyBERT approach described in [this paper](h
 We have performed intermediate layer distillation with roberta-base as the teacher which resulted in [deepset/tinyroberta-6l-768d](https://huggingface.co/deepset/tinyroberta-6l-768d).
 This model has not been distilled for any specific task. If you are interested in using distillation to improve its performance on a downstream task, you can take advantage of haystack's new [distillation functionality](https://haystack.deepset.ai/guides/model-distillation). You can also check out [deepset/tinyroberta-squad2](https://huggingface.co/deepset/tinyroberta-squad2) for a model that is already distilled on an extractive QA downstream task.
 
-## Usage
-
-### In Haystack
-Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. 
-To load and run the model with [Haystack](https://github.com/deepset-ai/haystack/):
-```python
-# After running pip install haystack-ai "transformers[torch,sentencepiece]"
-
-from haystack import Document
-from haystack.components.readers import ExtractiveReader
-
-docs = [
-    Document(content="Python is a popular programming language"),
-    Document(content="python ist eine beliebte Programmiersprache"),
-]
-
-reader = ExtractiveReader(model="deepset/tinyroberta-6l-768d")
-reader.warm_up()
-
-question = "What is a popular programming language?"
-result = reader.run(query=question, documents=docs)
-# {'answers': [ExtractedAnswer(query='What is a popular programming language?', score=0.5740374326705933, data='python', document=Document(id=..., content: '...'), context=None, document_offset=ExtractedAnswer.Span(start=0, end=6),...)]}
-```
-For a complete example with an extractive question answering pipeline that scales over many documents, check out the [corresponding Haystack tutorial](https://haystack.deepset.ai/tutorials/34_extractive_qa_pipeline).
-
-### In Transformers
-```python
-from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
-
-model_name = "deepset/tinyroberta-6l-768d"
-
-# a) Get predictions
-nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
-QA_input = {
-    'question': 'Why is model conversion important?',
-    'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
-}
-res = nlp(QA_input)
-
-# b) Load model & tokenizer
-model = AutoModelForQuestionAnswering.from_pretrained(model_name)
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-```
-
 ## About us
 <div class="grid lg:grid-cols-2 gap-x-4 gap-y-3">
     <div class="w-full h-40 object-cover mb-2 rounded-lg flex items-center justify-center">