text: 142,65,1361663915072118784,806058672619212800,Guillaume Lample,"New paper on code de-obfuscation: <LINK> We show that if you obfuscate the name of identifiers in source code, a model can retrieve the original names with very high accuracy. It even works when you remove the name of each variable / function! 1/3 <LINK> Retrieving masked variable names is much more difficult than retrieving randomly masked tokens, and turns out to be an excellent pretraining objective, that significantly outperforms Masked LM in unsupervised code translation and language code search. 2/3 <LINK> Also very useful if you try to understand other people's code when it is written with uninformative variable names! With @b_roziere @MaLachaux @MarcSzafraniec 3/3 <LINK>",https://arxiv.org/abs/2102.07492,"Recent advances in self-supervised learning have dramatically improved the state of the art on a wide variety of tasks. However, research in language model pre-training has mostly focused on natural languages, and it is unclear whether models like BERT and its variants provide the best pre-training when applied to other modalities, such as source code. In this paper, we introduce a new pre-training objective, DOBF, that leverages the structural aspect of programming languages and pre-trains a model to recover the original version of obfuscated source code. We show that models pre-trained with DOBF significantly outperform existing approaches on multiple downstream tasks, providing relative improvements of up to 13% in unsupervised code translation, and 24% in natural language code search. Incidentally, we found that our pre-trained model is able to de-obfuscate fully obfuscated source files, and to suggest descriptive variable names. ",DOBF: A Deobfuscation Pre-Training Objective for Programming Languages,3,"['New paper on code de-obfuscation: <LINK>\nWe show that if you obfuscate the name of identifiers in source code, a model can retrieve the original names with very high accuracy. It even works when you remove the name of each variable / function! 1/3 <LINK>', 'Retrieving masked variable names is much more difficult than retrieving randomly masked tokens, and turns out to be an excellent pretraining objective, that significantly outperforms Masked LM in unsupervised code translation and language code search. 2/3 https://t.co/7l8VWDP9OW', ""Also very useful if you try to understand other people's code when it is written with uninformative variable names!\nWith @b_roziere @MaLachaux @MarcSzafraniec 3/3 https://t.co/NF0qG68REm""]",21,02,687