text: 150,2,1037394871718301696,16495124,Sam Thomson,"WFSAs are dead. Long live WFSAs! Our new EMNLP paper ""Rational Recurrences"" is out: <LINK> Work by Hao Peng, @royschwartz02, me, and @nlpnoah. We analyze several recent RNNs as finite-state automata with neural transition weights. <LINK> These include quasi-RNNs (Bradbury et al), strongly-typed RNNs (Balduzzi & Ghifary), simple recurrent units (Lei et al), structurally constrained RNNs (Mikolov et al), input switched affine networks (Foerster et al)... *hand-waving* forget gates are self-loops!",https://arxiv.org/abs/1808.09357,"Despite the tremendous empirical success of neural models in natural language processing, many of them lack the strong intuitions that accompany classical machine learning approaches. Recently, connections have been shown between convolutional neural networks (CNNs) and weighted finite state automata (WFSAs), leading to new interpretations and insights. In this work, we show that some recurrent neural networks also share this connection to WFSAs. We characterize this connection formally, defining rational recurrences to be recurrent hidden state update functions that can be written as the Forward calculation of a finite set of WFSAs. We show that several recent neural models use rational recurrences. Our analysis provides a fresh view of these models and facilitates devising new neural architectures that draw inspiration from WFSAs. We present one such model, which performs better than two recent baselines on language modeling and text classification. Our results demonstrate that transferring intuitions from classical models like WFSAs can be an effective approach to designing and understanding neural models. ",Rational Recurrences,3,"['WFSAs are dead. Long live WFSAs!\n\nOur new EMNLP paper ""Rational Recurrences"" is out: <LINK>\nWork by Hao Peng, @royschwartz02, me, and @nlpnoah. We analyze several recent RNNs as finite-state automata with neural transition weights. <LINK>', 'These include quasi-RNNs (Bradbury et al), strongly-typed RNNs (Balduzzi &amp; Ghifary), simple recurrent units (Lei et al), structurally constrained RNNs (Mikolov et al), input switched affine networks (Foerster et al)...', '*hand-waving* forget gates are self-loops!']",18,08,499