text: 84,101,1438197765280919554,1182194132309012481,Michihiro Yasunaga,"Excited to share our new #EMNLP2021 paper ""LM-Critic: Language Models for Unsupervised Grammatical Error Correction"" with @percyliang @jure @StanfordAILab @StanfordNLP! Paper: <LINK> Github: <LINK> Thread below [1/7] ⤵️ <LINK> A big challenge in learning grammatical error correction (GEC) is to get labeled bad-good sentence pairs. Manual labeling is expensive. Synthetic data (e.g. perturbing good sentences) does not reflect real errors. How can we get cheap yet realistic training data for GEC? [2/7] <LINK> We develop a new solution to this problem leveraging language models (LMs). Two core insights: 1) LM-Critic: We can get an unsupervised critic that assesses the grammaticality of a sentence by checking if LMs assign it a higher probability than its local perturbations. [3/7] <LINK> How BIFI works is to jointly train the desired fixer that corrects a bad sentence into a good one and a breaker that corrupts a good sentence into a bad one, while using them in conjunction to generate more realistic parallel data from unlabeled raw text. [5/7] <LINK> In summary, using pretrained LMs, our method (LM-Critic + BIFI) can obtain usable parallel data from unlabeled data alone! We use the generated parallel data to augment the training of GEC models. This offers substantial performance boost across various GEC benchmarks. [6/7] <LINK> For more details, please check out our paper at <LINK> Huge thanks to the collaborators and all who gave us feedback! [7/7]",https://arxiv.org/abs/2109.06822,"Training a model for grammatical error correction (GEC) requires a set of labeled ungrammatical / grammatical sentence pairs, but manually annotating such pairs can be expensive. Recently, the Break-It-Fix-It (BIFI) framework has demonstrated strong results on learning to repair a broken program without any labeled examples, but this relies on a perfect critic (e.g., a compiler) that returns whether an example is valid or not, which does not exist for the GEC task. In this work, we show how to leverage a pretrained language model (LM) in defining an LM-Critic, which judges a sentence to be grammatical if the LM assigns it a higher probability than its local perturbations. We apply this LM-Critic and BIFI along with a large set of unlabeled sentences to bootstrap realistic ungrammatical / grammatical pairs for training a corrector. We evaluate our approach on GEC datasets across multiple domains (CoNLL-2014, BEA-2019, GMEG-wiki and GMEG-yahoo) and show that it outperforms existing methods in both the unsupervised setting (+7.7 F0.5) and the supervised setting (+0.5 F0.5). ",LM-Critic: Language Models for Unsupervised Grammatical Error Correction,6,"['Excited to share our new #EMNLP2021 paper ""LM-Critic: Language Models for Unsupervised Grammatical Error Correction"" with @percyliang @jure @StanfordAILab @StanfordNLP!\n\nPaper: <LINK>\nGithub: <LINK>\n\nThread below [1/7] ⤵️ <LINK>', 'A big challenge in learning grammatical error correction (GEC) is to get labeled bad-good sentence pairs. Manual labeling is expensive. Synthetic data (e.g. perturbing good sentences) does not reflect real errors. \nHow can we get cheap yet realistic training data for GEC?\n\n[2/7] https://t.co/2l97ok3rXA', 'We develop a new solution to this problem leveraging language models (LMs). Two core insights:\n\n1) LM-Critic: We can get an unsupervised critic that assesses the grammaticality of a sentence by checking if LMs assign it a higher probability than its local perturbations.\n\n[3/7] https://t.co/2O0qJ4NEVN', 'How BIFI works is to jointly train the desired fixer that corrects a bad sentence into a good one and a breaker that corrupts a good sentence into a bad one, while using them in conjunction to generate more realistic parallel data from unlabeled raw text.\n[5/7] https://t.co/alhh7dngYV', 'In summary, using pretrained LMs, our method (LM-Critic + BIFI) can obtain usable parallel data from unlabeled data alone!\n\nWe use the generated parallel data to augment the training of GEC models. This offers substantial performance boost across various GEC benchmarks.\n\n[6/7] https://t.co/Y0twpCydPf', 'For more details, please check out our paper at https://t.co/Fvi34s4LuQ\n\nHuge thanks to the collaborators and all who gave us feedback!\n[7/7]']",21,09,1470