Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.0.3] - 2021-11-01

Fixed

Nystrom causal attention [#75]

[0.0.2] - 2021-11-01

Fixed

More robust blocksparse [#24]

Added

Rotary embeddings [#32]
More flexible layernorm [#50]