File size: 1,573 Bytes
dd48e4a ed6d0e7 dd48e4a ed6d0e7 dd48e4a 954adf8 dd48e4a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
# Model card for CoNN Sub Carry
### Introduction
In paper Neural Comprehension: Language Models with Compiled Neural Networks , we introduced the integration of Compiled Neural Networks (CoNN) into the framework of language models, enabling existing language models to perform symbolic operations with perfect accuracy without the need for external tools.
In this model card, we introduce the Add Carry model, which is similar to the Transformer model and can perform borrow operations on a sequence of numbers subtracted.
### Install
```
git clone https://github.com/WENGSYX/Neural-Comprehension
cd Neural-Comprehension
pip install .
```
To run neural comprehension, you need to install `PyTorch`, `Transformers`, `jax`, and `tracr`.
### How to Use?
```
from NeuralCom.CoNN.modeling_conn import CoNNModel
from NeuralCom.CoNN import Tokenizer
model = CoNNModel.from_pretrained('WENGSYX/CoNN_Sub_Carry')
tokenizer = Tokenizer(model.config.input_encoding_map, model.config.output_encoding_map,model.config.max_position_embeddings)
output = model(tokenizer('2 -2 3 -9 4 0 8').unsqueeze(0))
>>> [['bos', '1', '8', '2', '1', '4', '0', '8']]
```
### 🙏Cite🙏
###### If you are interested in our paper, please feel free to cite it.
```
@misc{weng2023neural,
title={Neural Comprehension: Language Models with Compiled Neural Networks},
author={Yixuan Weng and Minjun Zhu and Fei Xia and Bin Li and Shizhu He and Kang Liu and Jun Zhao},
year={2023},
eprint={2304.01665},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
``` |