Pythia 160M, Mamba 130M, and RWKV 169M models trained on OpenWebText for 4000 steps (context window: 1024; effective batch size: 512). 6 seeds each.
James Michaelov
jmichaelov
AI & ML interests
None yet
Recent Activity
published
a dataset
3 days ago
jmichaelov/blimp_nl
updated
a dataset
3 days ago
jmichaelov/blimp_nl
published
a dataset
17 days ago
jmichaelov/lm_syneval
Organizations
models
43
jmichaelov/pythia-160m-512k-10mtok-mr95-decay90
Updated
jmichaelov/pythia-160m-512k-10mtok-mr10-decay10
Updated
jmichaelov/pythia-160m-512k-10mtok-mr10-decay20
Updated
jmichaelov/pythia-160m-512k-10mtok-mr0-decay100
Updated
jmichaelov/pythia-160m-512k-10mtok-mr10-decay80
Updated
jmichaelov/pythia-160m-512k-10mtok-mr10-decay40
Updated
jmichaelov/pythia-160m-512k-10mtok-mr20-decay10
Updated
jmichaelov/pythia-160m-512k-10mtok-mr40-decay40
Updated
jmichaelov/pythia-160m-512k-10mtok-mr40-decay20
Updated
jmichaelov/pythia-160m-512k-10mtok-mr80-decay10
Updated
datasets
12
jmichaelov/blimp_nl
Viewer
•
Updated
•
8.4k
•
99
jmichaelov/lm_syneval
Viewer
•
Updated
•
158k
•
29
jmichaelov/inverse_scaling_prize-hindsight_neglect
Viewer
•
Updated
•
315
•
13
jmichaelov/inverse_scaling_prize-memo_trap
Viewer
•
Updated
•
936
•
8
jmichaelov/inverse_scaling_prize-neqa
Viewer
•
Updated
•
300
•
8
jmichaelov/inverse_scaling_prize-redefine
Viewer
•
Updated
•
1.24k
•
3
jmichaelov/inverse_scaling_prize-into_the_unknown
Viewer
•
Updated
•
1.82k
•
3
jmichaelov/inverse_scaling_prize-modus_tollens
Viewer
•
Updated
•
1.24k
•
5
jmichaelov/inverse_scaling_prize-pattern_matching_suppression
Viewer
•
Updated
•
1.43k
•
4
jmichaelov/inverse_scaling_prize-repetitive_algebra
Viewer
•
Updated
•
1k
•
14