---
license: apache-2.0
# inference: false

# inference:
#   parameters:
tags:
- classification
- zero-shot
---

# Erlangshen-UniMC-Albert-235M-English
- Paper: [Zero-Shot Learners for Nature Language Understanding via a Unified Multiple Choice Perspective](https://arxiv.org/abs/2210.08590)
- Github: [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen/examples/unimc/)
- Docs: [Fengshenbang-Docs](https://fengshenbang-doc.readthedocs.io/)

## 简介 Brief Introduction

将自然语言理解任务转化为multiple choice任务，并且使用14个机器阅读理解数据进行预训练

Convert natural language understanding tasks into multiple choice tasks, and use 14 machine reading comprehension data for pre-training

## 模型分类 Model Taxonomy

|  需求 Demand  | 任务 Task       | 系列 Series      | 模型 Model    | 参数 Parameter | 额外 Extra |
|  :----:  | :----:  | :----:  | :----:  | :----:  | :----:  |
| 通用 General | 自然语言理解 NLU | 二郎神 Erlangshen | Albert |     235M    |     English     |

## 模型信息 Model Information

我们为零样本学习者提出了一种与输入无关的新范式，从某种意义上说，它与任何格式兼容并适用于一系列语言任务，例如文本分类、常识推理、共指解析、情感分析。我们的方法将零样本学习转化为多项选择任务，避免常用的大型生成模型（如 FLAN）中的问题。它不仅增加了模型的泛化能力，而且显着减少了对参数的需求。我们证明了这种方法可以在通用语言基准上取得最先进的性能，并在自然语言推理和文本分类等任务上产生令人满意的结果。更多详细信息可以参考我们的[论文](https://arxiv.org/abs/2210.08590)或者[GitHub](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen/examples/unimc/)

We propose an new paradigm for zero-shot learners that is input-agnostic, in the sense that  it is compatible with any format and applicable to a list of language tasks, such as text classification, commonsense  reasoning, coreference resolution, sentiment analysis.
Our approach converts zero-shot learning into multiple choice tasks, 
avoiding problems in commonly used large generative models such as FLAN. It not only adds generalization ability to the models, but also reduces the needs of parameters significantly. We demonstrate that this approach leads to state-of-the-art performance on common language benchmarks, and produces satisfactory results on tasks such as natural language inference and text classification.For more details, please refer to our [paper](https://arxiv.org/abs/2210.08590) or [github](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen/examples/unimc/)

### 下游效果 Performance

**Zero-Shot Classification**

| Model   | T0 11B | GLaM 60B | FLAN 137B | PaLM 540B | UniMC 235M |
|---------|--------|----------|-----------|-----------|------------|
| ANLI R1 | 43.6   | 40.9     | 47.7      | 48.4      | 52         |
| ANLI R2 | 38.7   | 38.2     | 43.9      | 44.2      | 44.4       |
| ANLI R3 | 41.3   | 40.9     | 47        | 45.7      | 47.8       |
| CB      | 70.1   | 33.9     | 64.1      | 51.8      | 75.7       |

## 使用 Usage

```python3
import argparse
from fengshen.pipelines.multiplechoice import UniMCPiplines

total_parser = argparse.ArgumentParser("TASK NAME")
total_parser = UniMCPiplines.piplines_args(total_parser)
args = total_parser.parse_args()
    
pretrained_model_path = 'IDEA-CCNL/Erlangshen-UniMC-Albert-235M-English'
args.language='english'
args.learning_rate=2e-5
args.max_length=512
args.max_epochs=3
args.batchsize=8
args.default_root_dir='./'
model = UniMCPiplines(args, model_path=pretrained_model_path)

train_data = [] 
dev_data = [] 
test_data = [{
	"texta": "it 's just incredibly dull .",
	"textb": "",
	"question": "What is sentiment of follow review?",
	"choice": ["it's great", "it's terrible"],
	"answer": "",
	"label": 0,
	"id": 19
}]

if args.train:
	model.train(train_data, dev_data)
result = model.predict(test_data)
```

## 引用 Citation

如果您在您的工作中使用了我们的模型，可以引用我们的[论文](https://arxiv.org/abs/2210.08590)：

If you are using the resource for your work, please cite the our [paper](https://arxiv.org/abs/2210.08590):

```text
@article{unimc,
  author    = {Ping Yang and
               Junjie Wang and
               Ruyi Gan and
               Xinyu Zhu and
               Lin Zhang and
               Ziwei Wu and
               Xinyu Gao and
               Jiaxing Zhang and
               Tetsuya Sakai},
  title     = {Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective},
  journal   = {CoRR},
  volume    = {abs/2210.08590},
  year      = {2022}
}
```

也可以引用我们的[网站](https://github.com/IDEA-CCNL/Fengshenbang-LM/):

You can also cite our [website](https://github.com/IDEA-CCNL/Fengshenbang-LM/):

```text
@misc{Fengshenbang-LM,
  title={Fengshenbang-LM},
  author={IDEA-CCNL},
  year={2021},
  howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
}
```