text:   Language Processing",6,"['Happy to share our new preprint (with @anmarasovic) “Teach Me to Explain: A Review of Datasets for Explainable NLP”\nPaper: <LINK>\nWebsite: <LINK>\n\nIt’s half survey, half reflections for more standardized ExNLP dataset collection. Highlights:\n\n1/6', 'We focus on datasets of the form: (inputs, labels, explanations). We describe these instance-wise explanations as “explaining human decisions” (the labels). Other types of explanations may explain something about the world. We focus on the shaded area.\n\n2/6 https://t.co/EUxVH09c8U', 'We identify 3 major classes of explanation datasets: highlights, free-text, and structured explanations. Each one is surveyed in a table like this (structured explanations here).\n\n&gt;95% of the datasets are collected using human annotation, and &gt;70% use crowdsourcing.\n\n3/6 https://t.co/jduqxNq174', 'We discuss how constraints placed on data annotation can influence modeling and evaluation, and suggest the use of datasheets to make collection decisions more transparent.\n\nAlso discuss structure that sometimes emerges free-text rationales and what to do about it.\n\n4/6 https://t.co/G6u3nvQrrC', 'Finally, we synthesize crowdsourcing methods from NLP/ HCI literature on improving quality+diversity of ExNLP datasets, such as using a crowd editing stage, collecting large set of explanations per instance, and using a diverse set of crowdworkers to avoid annotator bias.\n\n5/6', 'Please submit an issue or PR to our Github repository (linked through website) to add or edit characteristics of datasets! We are open to feedback.\n\n6/6']",21,02,1496