File size: 1,210 Bytes
7ee36a4 2940b53 7ee36a4 c7bd6ea 7ee36a4 6132f2f 7ee36a4 6132f2f 7ee36a4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
---
model-index:
- name: CodeLlama-7b
results:
- task:
type: code-generation
dataset:
name: Humaneval
type: humaneval
metrics:
- type: pass@1 (BASELINE)
value: 0.3048780487804878
- type: pass@1 (BASIC)
value: 0.3170731707317073
---
This is a d-Matrix functional reference of the CODELLAMA-7B model.
The reference provides the following functional *configurations*:
Configuration | Explanation
:-- | :--
**`BASELINE`** | a reference functionally equivalent to the original model
**`BASIC`** | all linear algebraic operands quantized to `MXINT8-64`, and all other operations transformed to approximated kernel simulations
### Usage
Install d-Matrix [Dmx_Compressor](https://github.com/d-matrix-ai/dmx-compressor) first.
```sh
pip install dmx_compressor
```
The following is an example model and its evaluation.
```sh
git clone https://github.com/bigcode-project/bigcode-evaluation-harness.git
cd bigcode-evaluation-harness
pip install -e .
```
```python
# code_eval.py included in HuggingFace repo
from code_eval import evaluate_model
eval_results = evaluate_model("d-matrix/CodeLlama-7b", dmx_config="BASELINE") # For BASELINE configuration
``` |