text: 432,59,1329481995542552579,1240355312,Yu Su,"GrailQA: A new large-scale, high-quality dataset for QA on knowledge bases - 64K questions over 3.7K relations across 86 domains - Support evaluation of i.i.d., compositional, and zero-shot generalization paper: <LINK> leaderboard: <LINK> My student @yugu_nlp will discuss the work at the Interactive and Executable Semantic Parsing Workshop@EMNLP (@intexsempar2020) at 23:00 (UTC) / 18:00 (EST). Joint work with Sue Kase, Michelle Vanni, Brian Sadler, @percyliang, Xifeng Yan We also propose new BERT-based KBQA models and demonstrate, for the first time, the critical role of joint contextualized representation of utterances and KB schema for non-i.i.d. generalization of KBQA models! More details in the paper.",https://arxiv.org/abs/2011.07743,"Existing studies on question answering on knowledge bases (KBQA) mainly operate with the standard i.i.d assumption, i.e., training distribution over questions is the same as the test distribution. However, i.i.d may be neither reasonably achievable nor desirable on large-scale KBs because 1) true user distribution is hard to capture and 2) randomly sample training examples from the enormous space would be highly data-inefficient. Instead, we suggest that KBQA models should have three levels of built-in generalization: i.i.d, compositional, and zero-shot. To facilitate the development of KBQA models with stronger generalization, we construct and release a new large-scale, high-quality dataset with 64,331 questions, GrailQA, and provide evaluation settings for all three levels of generalization. In addition, we propose a novel BERT-based KBQA model. The combination of our dataset and model enables us to thoroughly examine and demonstrate, for the first time, the key role of pre-trained contextual embeddings like BERT in the generalization of KBQA. ","Beyond I.I.D.: Three Levels of Generalization for Question Answering on