syntaxgym / syntaxgym.py

Commit History

interface change: use namedtuple for metric result so that we can add an `accuracy` property and have dot-access
e00b8f2

jgauthier commited on

update metric to support new dataset repr, and evaluating multiple suites in one call
5cd2907

jgauthier commited on

minor code cleanup
27bb1ab

jgauthier commited on

disable truncation. longer sequences are handled just fine, AFAICS
af46379

jgauthier commited on

add tokenizer config from perplexity metric. truncation breaks tests
4bd2962

jgauthier commited on

remove useless detach call
af32043

jgauthier commited on

move metric and tests from dataset repo
bcb8ccf

jgauthier commited on

stage template files
9a11e1b

jgauthier commited on