text: 378,17,1365057786250489858,1882939814,Sarah Wiegreffe,"Happy to share our new preprint (with @anmarasovic) “Teach Me to Explain: A Review of Datasets for Explainable NLP” Paper: <LINK> Website: <LINK> It’s half survey, half reflections for more standardized ExNLP dataset collection. Highlights: 1/6 We focus on datasets of the form: (inputs, labels, explanations). We describe these instance-wise explanations as “explaining human decisions” (the labels). Other types of explanations may explain something about the world. We focus on the shaded area. 2/6 <LINK> We identify 3 major classes of explanation datasets: highlights, free-text, and structured explanations. Each one is surveyed in a table like this (structured explanations here). &gt;95% of the datasets are collected using human annotation, and &gt;70% use crowdsourcing. 3/6 <LINK> We discuss how constraints placed on data annotation can influence modeling and evaluation, and suggest the use of datasheets to make collection decisions more transparent. Also discuss structure that sometimes emerges free-text rationales and what to do about it. 4/6 <LINK> Finally, we synthesize crowdsourcing methods from NLP/ HCI literature on improving quality+diversity of ExNLP datasets, such as using a crowd editing stage, collecting large set of explanations per instance, and using a diverse set of crowdworkers to avoid annotator bias. 5/6 Please submit an issue or PR to our Github repository (linked through website) to add or edit characteristics of datasets! We are open to feedback. 6/6",https://arxiv.org/abs/2102.12060,"Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual explanations. These explanations are used downstream in three ways: as data augmentation to improve performance on a predictive task, as supervision to train models to produce explanations for their predictions, and as a ground-truth to evaluate model-generated explanations. In this review, we identify 65 datasets with three predominant classes of textual explanations (highlights, free-text, and structured), organize the literature on annotating each type, identify strengths and shortcomings of existing collection methodologies, and give recommendations for collecting ExNLP datasets in the future. ","Teach Me to Explain: A Review of Datasets for Explainable Natural