Update README.md

README.md

@@ -1,144 +1,124 @@
 ---
-base_model: WhereIsAI/pre-UAE-Medical-Large-V1
-datasets: []
-language: []
+license: mit
+base_model: microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext
+model-index:
+- name: WhereIsAI/pubmed-angle-base-en
+  results: []
+datasets:
+- WhereIsAI/medical-triples
+- WhereIsAI/pubmedqa-test-angle-format-a
+- qiaojin/PubMedQA
+- ncbi/pubmed
+language:
+- en
 library_name: sentence-transformers
-pipeline_tag: sentence-similarity
-tags:
-- sentence-transformers
-- sentence-similarity
-- feature-extraction
-widget: []
 ---
 
-# SentenceTransformer based on WhereIsAI/pre-UAE-Medical-Large-V1
+# WhereIsAI/pubmed-angle-base-en
 
-This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [WhereIsAI/pre-UAE-Medical-Large-V1](https://huggingface.co/WhereIsAI/pre-UAE-Medical-Large-V1). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+This model is an example model for the Chinese blog post [【coming soon】](#) and [angle tutorial](https://angle.readthedocs.io/en/latest/notes/tutorial.html#tutorial). 
 
-## Model Details
+It was fine-tuned with [AnglE Loss](https://arxiv.org/abs/2309.12871) using the official [angle-emb](https://github.com/SeanLee97/AnglE).
 
-### Model Description
-- **Model Type:** Sentence Transformer
-- **Base model:** [WhereIsAI/pre-UAE-Medical-Large-V1](https://huggingface.co/WhereIsAI/pre-UAE-Medical-Large-V1) <!-- at revision c989d8965d489e9a6e873eabce06e6ef6f2a0188 -->
-- **Maximum Sequence Length:** 512 tokens
-- **Output Dimensionality:** 1024 tokens
-- **Similarity Function:** Cosine Similarity
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
+Related model: [WhereIsAI/pubmed-angle-base-en](https://huggingface.co/WhereIsAI/pubmed-angle-base-en)
 
-### Model Sources
+**1. Training Setup:**
 
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+- Base model: [microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext](https://huggingface.co/microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext)
+- Training Data: [WhereIsAI/medical-triples](https://huggingface.co/datasets/WhereIsAI/medical-triples), processed from [PubMedQA](https://huggingface.co/datasets/qiaojin/PubMedQA).
+- Test Data: [WhereIsAI/pubmedqa-test-angle-format-a](https://huggingface.co/datasets/WhereIsAI/pubmedqa-test-angle-format-a)
 
-### Full Model Architecture
+**2. Performance:**
 
-```
-SentenceTransformer(
-  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
-  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
-)
-```
+| Model                                  | Pooling Strategy | Spearman's Correlation |
+|----------------------------------------|------------------|:----------------------:|
+| tavakolih/all-MiniLM-L6-v2-pubmed-full | avg              | 84.56                  |
+| NeuML/pubmedbert-base-embeddings       | avg              | 84.88                  |
+| WhereIsAI/pubmed-angle-base-en     | cls              | 86.01                  |
+| **WhereIsAI/pubmed-angle-large-en**        | cls              | 86.21                  |
 
-## Usage
+**3. Citation**
 
-### Direct Usage (Sentence Transformers)
+Cite AnglE following 👉 https://huggingface.co/WhereIsAI/pubmed-angle-base-en#citation
 
-First install the Sentence Transformers library:
 
-```bash
-pip install -U sentence-transformers
-```
+## Usage
 
-Then you can load this model and run inference.
-```python
-from sentence_transformers import SentenceTransformer
+### via angle-emb
 
-# Download from the 🤗 Hub
-model = SentenceTransformer("WhereIsAI/pre-UAE-Medical-Large-V1")
-# Run inference
-sentences = [
-    'The weather is lovely today.',
-    "It's so sunny outside!",
-    'He drove to the stadium.',
-]
-embeddings = model.encode(sentences)
-print(embeddings.shape)
-# [3, 1024]
-
-# Get the similarity scores for the embeddings
-similarities = model.similarity(embeddings, embeddings)
-print(similarities.shape)
-# [3, 3]
+```bash
+python -m pip install -U angle-emb
 ```
 
-<!--
-### Direct Usage (Transformers)
-
-<details><summary>Click to see the direct usage in Transformers</summary>
+Example:
 
-</details>
--->
+```python
+from angle_emb import AnglE
+from angle_emb.utils import cosine_similarity
 
-<!--
-### Downstream Usage (Sentence Transformers)
+# 1. load
+angle = AnglE.from_pretrained('WhereIsAI/pubmed-angle-large-en', pooling_strategy='cls').cuda()
 
-You can finetune this model on your own dataset.
+query = 'How to treat childhood obesity and overweight?'
+docs = [
+    query,
+    'The child is overweight. Parents should relieve their children\'s symptoms through physical activity and healthy eating. First, they can let them do some aerobic exercise, such as jogging, climbing, swimming, etc. In terms of diet, children should eat more cucumbers, carrots, spinach, etc. Parents should also discourage their children from eating fried foods and dried fruits, which are high in calories and fat. Parents should not let their children lie in bed without moving after eating. If their children\'s condition is serious during the treatment of childhood obesity, parents should go to the hospital for treatment under the guidance of a doctor in a timely manner.',
+    'If you want to treat tonsillitis better, you can choose some anti-inflammatory drugs under the guidance of a doctor, or use local drugs, such as washing the tonsil crypts, injecting drugs into the tonsils, etc. If your child has a sore throat, you can also give him or her some pain relievers. If your child has a fever, you can give him or her antipyretics. If the condition is serious, seek medical attention as soon as possible. If the medication does not have a good effect and the symptoms recur, the author suggests surgical treatment. Parents should also make sure to keep their children warm to prevent them from catching a cold and getting tonsillitis again.',
+]
 
-<details><summary>Click to expand</summary>
+# 2. encode
+embeddings = angle.encode(docs)
+query_emb = embeddings[0]
 
-</details>
--->
+for doc, emb in zip(docs[1:], embeddings[1:]):
+    print(cosine_similarity(query_emb, emb))
 
-<!--
-### Out-of-Scope Use
+# 0.8181731743429251
+# 0.43483792889514516
+```
 
-*List how the model may foreseeably be misused and address what users ought not to do with the model.*
--->
 
-<!--
-## Bias, Risks and Limitations
+### via sentence-transformers
 
-*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
--->
+Install sentence-transformers
 
-<!--
-### Recommendations
+```bash
+python -m pip install -U sentence-transformers
+```
 
-*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
--->
+```python
+from sentence_transformers import SentenceTransformer
+from sentence_transformers.util import cos_sim
 
-## Training Details
 
-### Framework Versions
-- Python: 3.10.12
-- Sentence Transformers: 3.0.1
-- Transformers: 4.42.3
-- PyTorch: 2.3.0+cu121
-- Accelerate: 0.30.1
-- Datasets: 2.19.1
-- Tokenizers: 0.19.1
+# 1. load model
+model = SentenceTransformer("WhereIsAI/pubmed-angle-large-en")
 
-## Citation
+query = 'How to treat childhood obesity and overweight?'
+docs = [
+    query,
+    'The child is overweight. Parents should relieve their children\'s symptoms through physical activity and healthy eating. First, they can let them do some aerobic exercise, such as jogging, climbing, swimming, etc. In terms of diet, children should eat more cucumbers, carrots, spinach, etc. Parents should also discourage their children from eating fried foods and dried fruits, which are high in calories and fat. Parents should not let their children lie in bed without moving after eating. If their children\'s condition is serious during the treatment of childhood obesity, parents should go to the hospital for treatment under the guidance of a doctor in a timely manner.',
+    'If you want to treat tonsillitis better, you can choose some anti-inflammatory drugs under the guidance of a doctor, or use local drugs, such as washing the tonsil crypts, injecting drugs into the tonsils, etc. If your child has a sore throat, you can also give him or her some pain relievers. If your child has a fever, you can give him or her antipyretics. If the condition is serious, seek medical attention as soon as possible. If the medication does not have a good effect and the symptoms recur, the author suggests surgical treatment. Parents should also make sure to keep their children warm to prevent them from catching a cold and getting tonsillitis again.',
+]
 
-### BibTeX
 
-<!--
-## Glossary
+# 2. encode
+embeddings = model.encode(docs)
 
-*Clearly define terms in order to be accessible across audiences.*
--->
+similarities = cos_sim(embeddings[0], embeddings[1:])
+print('similarities:', similarities)
+```
 
-<!--
-## Model Card Authors
 
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
+## Citation
 
-<!--
-## Model Card Contact
+If you use this model for academic purpose, please cite AnglE's paper, as follows:
 
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->
+```bibtext
+@article{li2023angle,
+  title={AnglE-optimized Text Embeddings},
+  author={Li, Xianming and Li, Jing},
+  journal={arXiv preprint arXiv:2309.12871},
+  year={2023}
+}
+```