# Open-source Softmax Linear Unit

Replicating the results in the paper [Softmax Linear Units](https://transformer-circuits.pub/2022/solu/index.html) published recently by Anthropic. 

TODO: update this readme with:
- usage information
- how to log runs, how to use pysvelte to look at activations