Upload 7 files

README.md

@@ -1,3 +1,77 @@
 ---
+language: en
+tags:
+- transformers
+- feature-extraction
+- materials
 license: other
 ---
+
+# PolymerNER
+
+This model is a fine-tuned version of the MaterialsBERT model on a dataset of 638 abstracts and contains a linear layer on top of MaterialsBERT to predict the entity type of each token. The entity types predicted by this model are POLYMER, POLYMER\_FAMILY, ORGANIC, INORGANIC, MONOMER, PROP\_NAME, PROP\_VALUE, MATERIAL\_AMOUNT
+This named entity recognition (NER) model was introduced in [this](https://www.nature.com/articles/s41524-023-01003-w) paper. Refer to this paper for a more detailed description of the entities and performance metrics of this model. As MaterialsBERT is uncased, the NER model is also uncased. 
+
+## Intended uses & limitations
+
+You can use the model for sequence labeling/entity tagging tasks on materials science text. The training, validation and test data for this model consisted of abstracts related to polymers. The entities tagged by this model however are general and can be used with any materials science text to tag the entity types defined in the ontology of this model.
+
+## How to Use
+
+Here is how to use this model to tag entities given some text:
+
+```python
+from transformers import AutoModelForTokenClassification, AutoTokenizer, pipeline
+tokenizer = AutoTokenizer.from_pretrained('pranav-s/PolymerNER', model_max_length=512)
+model = AutoModelForTokenClassification.from_pretrained('pranav-s/PolymerNER')
+ner_pipeline = pipeline(task="ner", model=model, tokenizer=tokenizer, aggregation_strategy="simple", device='cpu')
+text = "Polyethylene has a glass transition temperature of -100 °C"
+ner_output = ner_pipeline(text)
+```
+
+## Training data
+
+A training data set of 638 polymer abstracts was used. The data set is provided [here](https://github.com/Ramprasad-Group/polymer_information_extraction)
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train\_batch_size: 8
+- eval\_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr\_scheduler_type: linear
+- num_epochs: 5
+
+
+### Framework versions
+
+- Transformers 4.17.0
+- Pytorch 1.10.2
+- Datasets 1.18.3
+- Tokenizers 0.11.0
+
+
+## Citation
+
+If you find PolymerNER useful in your research, please cite the following paper:
+
+```latex
+@article{materialsbert,
+  title={A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing},
+  author={Shetty, Pranav and Rajan, Arunkumar Chitteth and Kuenneth, Chris and Gupta, Sonakshi and Panchumarti, Lakshmi Prerana and Holm, Lauren and Zhang, Chao and Ramprasad, Rampi},
+  journal={npj Computational Materials},
+  volume={9},
+  number={1},
+  pages={52},
+  year={2023},
+  publisher={Nature Publishing Group UK London}
+}
+```
+
+<a href="https://huggingface.co/exbert/?model=pranav-s/PolymerNER">
+	<img width="300px" src="https://cdn-media.huggingface.co/exbert/button.png">
+</a>

config.json

@@ -0,0 +1,48 @@
+{
+  "_name_or_path": "/data/pranav/projects/matbert/pretrained_models/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext",
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "finetuning_task": "ner",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "INORGANIC",
+    "1": "MATERIAL_AMOUNT",
+    "2": "MONOMER",
+    "3": "O",
+    "4": "ORGANIC",
+    "5": "POLYMER",
+    "6": "POLYMER_FAMILY",
+    "7": "PROP_NAME",
+    "8": "PROP_VALUE"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "INORGANIC": 0,
+    "MATERIAL_AMOUNT": 1,
+    "MONOMER": 2,
+    "O": 3,
+    "ORGANIC": 4,
+    "POLYMER": 5,
+    "POLYMER_FAMILY": 6,
+    "PROP_NAME": 7,
+    "PROP_VALUE": 8
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.17.0.dev0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73c7ad087af05c9dbd3b8b99d4550a7b8e5eccf6c10efa7e445c505f579478ad
+size 435677681

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "special_tokens_map_file": null, "name_or_path": "/data/pranav/projects/matbert/pretrained_models/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext", "do_basic_tokenize": true, "never_split": null, "tokenizer_class": "BertTokenizer"}