don't mix up condition indexing in suites where items have different order of content by condition (e.g. number_prep in syntaxgym2020) 7eae31f jgauthier commited on Nov 1, 2022
get_region_edges: fix edge case (ha) where empty initial regions cause the count to be off 8059baf jgauthier commited on Aug 12, 2022
interface change: use namedtuple for metric result so that we can add an `accuracy` property and have dot-access e00b8f2 jgauthier commited on Jul 8, 2022
update metric to support new dataset repr, and evaluating multiple suites in one call 5cd2907 jgauthier commited on Jul 8, 2022
disable truncation. longer sequences are handled just fine, AFAICS af46379 jgauthier commited on Jul 7, 2022
add tokenizer config from perplexity metric. truncation breaks tests 4bd2962 jgauthier commited on Jul 7, 2022