Update README.md

README.md

@@ -41,7 +41,7 @@ We introduce LEGAL-ROBERTA, which is a domain-specific language representation m
 | legalBert    |  torture    | detention    | arrest    |  rape   |   death       |
 | legalRoberta | torture     |  abuse   |  insanity   |   cruelty  |    confinement      |
 
-'Establishing a system for the identification and registration of \<mask\> animals and regarding the labelling of beef and beef products .':
+'Establishing a system for the identification and registration of \<mask\> animals and regarding the labeling of beef and beef products .':
 
 | Model        | top1 | top2    |  top3   | top4    | top5 |
 | ------------ | ---- | --- | --- | --- | -------- |
@@ -49,9 +49,18 @@ We introduce LEGAL-ROBERTA, which is a domain-specific language representation m
 | legalBert    |   live   |  beef   |  farm   | pet    |      dairy    |
 | legalRoberta | domestic     |  all   |  beef   |   wild  |    registered      |
 
+## Load Pretrained Model
+
+```python
+from transformers import AutoTokenizer, AutoModel
+
+tokenizer = AutoTokenizer.from_pretrained("saibo/legal-roberta-base")
+model = AutoModel.from_pretrained("saibo/legal-roberta-base")
+```
+
 ## Training data
 
-The tranining data consists of 3 origins:
+The training data consists of 3 origins:
 
 1. Patent Litigations (https://www.kaggle.com/uspto/patent-litigations): This dataset covers over 74k cases across 52 years and over 5 million relevant documents. 5 different files detail the litigating parties, their attorneys, results, locations, and dates.
     1. *1.57GB*
@@ -59,7 +68,7 @@ The tranining data consists of 3 origins:
     3. *clean 1.1GB*
 
 
-2. Caselaw Access Project (CAP) (https://case.law/): Following 360 years of United States caselaw, Caselaw Access Project (CAP) API and bulk data services includes 40 million pages of U.S. court decisions and almost 6.5 million individual cases.
+2. Caselaw Access Project (CAP) (https://case.law/): Following 360 years of United States case law, Caselaw Access Project (CAP) API and bulk data services includes 40 million pages of U.S. court decisions and almost 6.5 million individual cases.
     1. *raw 5.6*
     2. abbrev:CAP
     3. *clean 2.8GB*
@@ -69,7 +78,7 @@ The tranining data consists of 3 origins:
     3. *clean 1GB*
 
 ## Training procedure
-We start from a pretrained ROBERTA-BASE model and fine-tune it on legal corpus.
+We start from a pretrained ROBERTA-BASE model and fine-tune it on the legal corpus.
 
 Fine-tuning configuration:
 - lr = 5e-5(with lr decay, ends at 4.95e-8)
@@ -128,8 +137,8 @@ tranining time per epoch(including validation ):
 
 
 ## Limitations:
-In the Masked Language Model showroom, the tokens have a prefix **Ġ**. This seems to be wired but I haven't yet been able to fix it.
-I know in case of BPE tokenizer(ROBERTA's tokenizer), the symbol Ġ means the end of a new token and the majority of tokens in vocabs of pre-trained tokenizers start with Ġ.
+In the Masked Language Model showroom, the tokens have the prefix **Ġ**. This seems to be wired but I haven't yet been able to fix it.
+I know in the case of BPE tokenizer(ROBERTA's tokenizer), the symbol Ġ means the end of a new token, and the majority of tokens in the vocabs of pre-trained tokenizers start with Ġ.
 
 For example
 ```python
@@ -150,4 +159,3 @@ So I think this is not fundamentally linked to the model itself.
 
 
 
-