pszemraj commited on
Commit
d9189c0
·
1 Parent(s): 98ae69d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -11
README.md CHANGED
@@ -70,17 +70,26 @@ It achieves the following results on the evaluation set:
70
  - Loss: 3.7502
71
  - Accuracy: 0.3650
72
 
73
- ## Model description
74
-
75
- More information needed
76
-
77
- ## Intended uses & limitations
78
-
79
- More information needed
80
-
81
- ## Training and evaluation data
82
-
83
- More information needed
 
 
 
 
 
 
 
 
 
84
 
85
  ## Training procedure
86
 
 
70
  - Loss: 3.7502
71
  - Accuracy: 0.3650
72
 
73
+ ## eval
74
+
75
+ initial 'get the feet wet':
76
+
77
+ `hf-causal-experimental (pretrained=pszemraj/mega-ar-small-4096-sw_minipile,revision=main,trust_remote_code=True,dtype='float'), limit: None, provide_description: False, num_fewshot: 0, batch_size: 16`
78
+
79
+ | Task |Version| Metric | Value | | Stderr |
80
+ |--------------|------:|--------|--------:|---|-------:|
81
+ |arc_easy | 0|acc | 0.3173|± | 0.0096|
82
+ | | |acc_norm| 0.3022|± | 0.0094|
83
+ |boolq | 1|acc | 0.4107|± | 0.0086|
84
+ |lambada_openai| 0|ppl |6843.1824|± |295.0792|
85
+ | | |acc | 0.0155|± | 0.0017|
86
+ |openbookqa | 0|acc | 0.1220|± | 0.0147|
87
+ | | |acc_norm| 0.2480|± | 0.0193|
88
+ |piqa | 0|acc | 0.5609|± | 0.0116|
89
+ | | |acc_norm| 0.5566|± | 0.0116|
90
+ |winogrande | 0|acc | 0.5059|± | 0.0141|
91
+
92
+ still some ways to go.
93
 
94
  ## Training procedure
95