text: 233,75,1426259065202692103,1425872270627639303,or castel,"*New #NLProc paper alert!*  We examine greedy decoding for extractive QA, by comparing it with our optimal algorithm: exact-extract. We found out that given just a few examples - greedy decoding does a great job! Paper: <LINK>  1/N Greedy decoding is used with great success for extractive QA. But it is not necessarily extractive (=generates spans from the passage) nor exact (=produces the max prob span). Can we do better? With @ori__ram @AviaEfrat and @omerlevy_  Summary Thread: 2/N We present a novel decoding algorithm: *exact-extract*, a dynamic programming algorithm that efficiently calculates the probability of all possible spans from the input passage, enabling us to find the most probable one. 3/N We first examine T5’s performance in a few-shot setting. Given only 16 examples, the model reaches ~82% F1 on SQuAD. 1024 examples are enough to reach human performance (91% F1); all this while using the naive greedy decoding, matching our optimal algorithm quite quickly.  4/N <LINK> On some datasets the gap is more substantial - but it shrinks down at 1024 examples too. 5/N <LINK> This is not the case for the zero-shot setting, where exact-extract obtains 60% F1 on SQuAD, without any fine-tuning (10 points over greedy decoding). 6/N We have plenty more analyses, results, and even a lightweight pretraining method for improving greedy decoding in zero-shot. 7/N Overall, our results show that the naive greedy decoding is nearly as good as the optimal strategy even when only a handful of labeled examples are available; and that pretrained language models can be easily adapted to extractive QA and with great results.  8/8 and the paper, since tweeter is hiding the first tweet🤨 : <LINK> and the paper, since tweeter is hiding the first tweet 🤨 : <LINK>",https://arxiv.org/abs/2108.05857,"Fine-tuned language models use greedy decoding to answer reading comprehension questions with relative success. However, this approach does not ensure that the answer is a span in the given passage, nor does it guarantee that it is the most probable one. Does greedy decoding actually perform worse than an algorithm that does adhere to these properties? To study the performance and optimality of greedy decoding, we present exact-extract, a decoding algorithm that efficiently finds the most probable answer span in the context. We compare the performance of T5 with both decoding algorithms on zero-shot and few-shot extractive question answering. When no training examples are available, exact-extract significantly outperforms greedy decoding. However, greedy decoding quickly converges towards the performance of exact-extract with the introduction of a few training examples, becoming more extractive and increasingly likelier to generate the most probable span as the training set grows. We also show that self-supervised training can bias the model towards extractive behavior, increasing performance in the zero-shot setting without resorting to annotated examples. Overall, our results suggest that pretrained language models are so good at adapting to extractive question answering, that it is often enough to fine-tune on a small training set for the greedy algorithm to emulate the optimal decoding strategy. ",How Optimal is Greedy Decoding for Extractive Question Answering?,10,"['*New #NLProc paper alert!*\n \nWe examine greedy decoding for extractive QA, by comparing it with our optimal algorithm: exact-extract. We found out that given just a few examples - greedy decoding does a great job!\nPaper: <LINK>\n \n1/N', 'Greedy decoding is used with great success for extractive QA. But it is not necessarily extractive (=generates spans from the passage) nor exact (=produces the max prob span). Can we do better?\n\nWith @ori__ram  @AviaEfrat  and @omerlevy_ \n\nSummary Thread:\n\n2/N', 'We present a novel decoding algorithm: *exact-extract*, a dynamic programming algorithm that efficiently calculates the probability of all possible spans from the input passage, enabling us to find the most probable one.\n\n3/N', 'We first examine T5’s performance in a few-shot setting. Given only 16 examples, the model reaches ~82% F1 on SQuAD. 1024 examples are enough to reach human performance (91% F1); all this while using the naive greedy decoding, matching our optimal algorithm quite quickly. \n\n4/N https://t.co/0msfiP5UCk', 'On some datasets the gap is more substantial - but it shrinks down at 1024 examples too.\n\n5/N https://t.co/EBQP9pj8vn', 'This is not the case for the zero-shot setting, where exact-extract obtains 60% F1 on SQuAD, without any fine-tuning (10 points over greedy decoding).\n\n6/N', 'We have plenty more analyses, results, and even a lightweight pretraining method for improving greedy decoding in zero-shot.\n\n7/N', 'Overall, our results show that the naive greedy decoding is nearly as good as the optimal strategy even when only a handful of labeled examples are available; and that pretrained language models can be easily adapted to extractive QA and with great results. \n\n8/8', 'and the paper, since tweeter is hiding the first tweet🤨 : https://t.co/Rfo8eIZs65', 'and the paper, since tweeter is hiding the first tweet 🤨 : https://t.co/Rfo8eIZs65']",21,08,1775