|
--- |
|
license: apache-2.0 |
|
--- |
|
## Model Card: dstc11-simmc2.1-iflytek |
|
|
|
## Recent Update |
|
- 👏🏻 2022.10.10: The repository `dstc11-simmc2.1-iflytek` for [DSTC11 Track1](https://github.com/facebookresearch/simmc2) is created. |
|
|
|
|
|
## Overview |
|
The [SIMMC2.1](https://github.com/facebookresearch/simmc2) challenge aims to lay the foundations for the real-world assistant agents that can handle multimodal inputs, and perform multimodal actions. It has 4 tasks: Ambiguous Candidate Identification, Multimodal Coreference Resolution, Multimodal Dialog State Tracking, Response Generation. We consider the joint input of textual context, tokenized objects and scene as multi-modal input, as well as compare the performance of single task training and multi task joint training. |
|
|
|
## Model Date |
|
Model was originally released in October 2022. |
|
|
|
## Model Type |
|
The **mt-bart**, **mt-bart-sys** and **mt-bart-sys-nvattr** have the same model framework (transformer with multi-task head), which are finetuned on [SIMMC2.1](https://github.com/facebookresearch/simmc2) based on the pretrained [BART-Large](https://huggingface.co/facebook/bart-large) model. This [repository](https://github.com/scutcyr/dstc11-simmc2.1-iflytek) also contains code to finetune the model. |
|
|
|
|
|
## Using with Transformers |
|
(1) You should first download the model from huggingface used the scripts: |
|
```bash |
|
cd ~ |
|
mkdir pretrained_model |
|
cd pretrained_model |
|
git lfs install |
|
git clone https://huggingface.co/scutcyr/dstc11-simmc2.1-iflytek |
|
``` |
|
(2) Then you should clone our code use the follow scripts: |
|
```bash |
|
cd ~ |
|
git clone https://github.com/scutcyr/dstc11-simmc2.1-iflytek.git |
|
``` |
|
(3) Follow the [README](https://github.com/scutcyr/dstc11-simmc2.1-iflytek#readme) to use the model. |
|
|
