Update README.md

README.md

@@ -1,12 +1,22 @@
 ---
 library_name: transformers
-tags: []
+tags:
+- lecture
+- college
+- university
+- summarization
+license: mit
+language:
+- en
+metrics:
+- rouge
+pipeline_tag: summarization
 ---
 
 # Model Card for Model ID
 
 <!-- Provide a quick summary of what the model is/does. -->
-
+Academ is a fine-tuned BART model for summarizing academic lectures. 
 
 
 ## Model Details
@@ -17,13 +27,13 @@ tags: []
 
 This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
 
-- **Developed by:** [More Information Needed]
+- **Developed by:** Yousef Gamaleldin
 - **Funded by [optional]:** [More Information Needed]
 - **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
+- **Model type:** Summarization
+- **Language(s) (NLP):** English
 - **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Finetuned from model [optional]:** BART Large
 
 ### Model Sources [optional]
 
@@ -41,6 +51,8 @@ This is the model card of a 🤗 transformers model that has been pushed on the
 
 <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
+
+
 [More Information Needed]
 
 ### Downstream Use [optional]
@@ -65,12 +77,35 @@ This is the model card of a 🤗 transformers model that has been pushed on the
 
 <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. More information needed for further recommendations.
 
 ## How to Get Started with the Model
 
 Use the code below to get started with the model.
 
+def get_summary(input_ids, attention_mask, context_length):
+    
+    summaries = []
+    for i in range(0, input_ids.shape[1], context_length):
+        
+        input_slice = input_ids[:, i:i + context_length] if i + context_length <= input_ids.size(1) else input_ids[:, i:]
+        attention_mask_slice = attention_mask[:, i:i + context_length] if i + context_length <= attention_mask.size(1) else attention_mask[:, i:]
+        
+        summary = model.generate(input_slice, attention_mask = attention_mask_slice, max_new_tokens = 1654, min_new_tokens = 250, do_sample = True, renormalize_logits = True)
+        summaries.extend(summary[0].tolist())
+        
+    summaries = tokenizer.decode(summaries, skip_special_tokens = True)
+    
+    return summaries
+    
+batch = tokenizer(texts, truncation = False) # make sure to get the transcript from the lecture
+
+input_ids = torch.tensor(batch['input_ids']).unsqueeze(0).to(device)
+attention_mask = torch.tensor(batch['attention_mask']).unsqueeze(0).to(device)
+
+summary = get_summary(input_ids, attention_mask, 1654)
+print(summary)
+
 [More Information Needed]
 
 ## Training Details
@@ -92,8 +127,11 @@ Use the code below to get started with the model.
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
+- **Training regime:** [More Information Needed] bf16 non-mixed precision<!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+- **Learning Rate:** 0.001
+- **Weight Decay:** 0.01
+- **Epochs:** 4
+- **Batch Size:** 16
 #### Speeds, Sizes, Times [optional]
 
 <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
@@ -103,11 +141,13 @@ Use the code below to get started with the model.
 ## Evaluation
 
 <!-- This section describes the evaluation protocols and provides the results. -->
+The evaluation is based on ROUGE 1 with a change of discounting padding tokens. 
 
 ### Testing Data, Factors & Metrics
 
 #### Testing Data
 
+The model's test dataset had 289 lectures, mainly from MIT OpenCourseWare.
 <!-- This should link to a Dataset Card if possible. -->
 
 [More Information Needed]
@@ -129,6 +169,7 @@ Use the code below to get started with the model.
 [More Information Needed]
 
 #### Summary
+Academ is a summarization model trained on 2307 lectures, mainly from MIT OpenCourseWare. The model has a max sequence length of 1654, an increase of 630 tokens from the original model.