README.md

---
license: apache-2.0
---

This repo offers a set of fairseq roberta based models fine tuned to specific NLP tasks.
These models are:
- Small
- Easy to host on T4 or V100
- 100x faster than using LLMs for similar tasks
- Easy to fine tune

All the models below were trained at Nlmatics Corp. from 2019-2023 with base model from: https://github.com/facebookresearch/fairseq/blob/main/examples/roberta/README.md 

### To run the models:
Use https://github.com/nlmatics/nlm-model-service

### To acccess the models 
Use https://github.com/nlmatics/nlm-utils 

### To train the models
TBD

## List of Models

Click on each model to see details:

### roberta.large.boolq

*Location:* [roberta.large.boolq](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.boolq)  

Trained with MNLI + Boolq

*Trained by:* Evan Li

*Application:* Given a passage and a question, answer the question with yes, no or unsure.

*Training Process:* https://blogs.nlmatics.com/2020/03/12/Boolean-Question-Answering-with-Neutral-Labels.html

### roberta.large.qa
See folder: [roberta.large.qa](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.qa)  

Trained with SQuAD 2.0 + Custom Dataset preferring shorter spans better suited for data extraction

*Trained by:* Ambika Sukla

*Application:* Given a passage and a question, pick the shortest span from the passage that answers the question

*Training Process:* start, end location head on the top of Roberta Base

### roberta.large.stsb
*See folder:* [roberta.large.stsb](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.stsb)

Trained with STSB dataset

*Trained by:* Meta/Fairseq

*Application:* Given two passages, return a score beteen 0 and 1 to evaluate their similarity

*Training Process:* regression head on top of Roberta Base

### roberta.large.phraseqa
*See folder:* [roberta.large.phraseqa](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.phraseqa)

Trained with Roberta 2.0 with the question words removed from the question

*Trained By:* Batya Stein

*Application:* Given a passage and phrase (key), extract a value from the passage 

*Training Process:* https://blogs.nlmatics.com/2020/08/25/Optimizing-Transformer-Q&A-Models-for-Naturalistic-Search.html

### roberta.large.qasrl

*See folder:* [roberta.large.qasrl](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.qasrl)

Trained with QASRL dataset

*Application:* Given a passage, get back values for who, what, when, where etc.

*Trained By:* Nima Sheikholeslami

### roberta.large.qatype.lower.RothWithQ

*See folder:* [roberta.large.qatype.lower.RothWithQ](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.qatype.lower.RothWithQ)

Trained with the Roth Question Type dataset.

*Application:* Given a question, return one of the answer types e.g. number, location. See the Roth dataset for full list. 

*Trained By:* Evan Li

### roberta.large.io_qa

See folder: [roberta.large.io_qa](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.io_qa)
Trained with SQuAD 2.0 dataset

*Trained By:* Nima Sheikholeslami

*Training Process:* Use io head to support multiple spans.