text: 97,42,1339979635359182850,2423179856,Edward Raff,"Used MalConv and annoyed with the huge training cost? Looked at it with disdain for being unable to learn feature interactions? With @willcfleshman @rjzak @drhyrum & @filar we have what you need: A new @RealAAAI #AAAI2021 paper📝<LINK> 🧑‍💻 <LINK> <LINK> MalConv up to 25x faster and over 100x+ more memory efficient with a new chunked approach to temporal max pooling. Max pooling's gradient is sparse, but no one ever exploits it because their problems are too small. With this we can now train on files with T=100,000,000+ steps! <LINK> We want feature interactions, but 100M+ steps is still too much for transformers. So we develop a new attention based gating mechanism ""Global Channel Gating"" (GCG) that allows learning interactions over 100M+ steps! <LINK> Using GCG we have MalConv2, which has a feature extraction and context extraction sub-networks, which interact through GCG to modulate parts of the input based on other content. <LINK> All of this work isn't getting us to domain knowledge levels yet, but a nice improvement work learning from raw bytes! We are also circumventing a trivial attack, we can process the _entire_ file even if its hundreds of MB. No trivial ""just append to the end"" evasion. <LINK> As mentioned, also way fafster! We are processing more data in less time and less RAM! The huge reductions is what allows us to re-invest those savings into MalConv2.0 <LINK> Impossible without @willcfleshman in particular, we can also look at how the GCG attention impacts what is/is-not used to make decisions. The results are pretty good, and seems to learn kind of intuitive logic an analyst might use. Hard to scale these kinds of studies up though. <LINK> Happy to get this work out there, and would not have been possible without so many people. Especially all the wonderful people who have used and attacked MalConv! Hearing about the successes and troubles in using it drove a lot of the thought behind this work. If MalConv2 is useful in anyway to you, please drop a line and let us know! Especially with this pandemic thing, my normal feedback network of conferences doesn't work as well as normal 🙃",https://arxiv.org/abs/2012.09390,"Recent works within machine learning have been tackling inputs of ever-increasing size, with cybersecurity presenting sequence classification problems of particularly extreme lengths. In the case of Windows executable malware detection, inputs may exceed $100$ MB, which corresponds to a time series with $T=100,000,000$ steps. To date, the closest approach to handling such a task is MalConv, a convolutional neural network capable of processing up to $T=2,000,000$ steps. The $\mathcal{O}(T)$ memory of CNNs has prevented further application of CNNs to malware. In this work, we develop a new approach to temporal max pooling that makes the required memory invariant to the sequence length $T$. This makes MalConv $116\times$ more memory efficient, and up to $25.8\times$ faster to train on its original dataset, while removing the input length restrictions to MalConv. We re-invest these gains into improving the MalConv architecture by developing a new Global Channel Gating design, giving us an attention mechanism capable of learning feature interactions across 100 million time steps in an efficient manner, a capability lacked by the original MalConv CNN. Our implementation can be found at this https URL ","Classifying Sequences of Extreme Length with Constant Memory Applied to