1.9.2

What's Changed

• remove duplicated qkv computation in na_vit_nested_tensor_3d.py by @JacobLinCool in #341

New Contributors

• @JacobLinCool made their first contribution in #341

Full Changelog: 1.9.1...1.9.2

1.9.1

1.9.1

1.9.0

Full Changelog: 1.8.9...1.9.0

1.8.9

Full Changelog: 1.8.8...1.8.9

1.8.8

allow for qk norm to be turned off for na vit nested tensor

1.8.7

fix value residual

1.8.6

add value residual based simple vit

1.8.5

Full Changelog: 1.8.4...1.8.5

1.8.4

go all the way with the normalized vit, fix some scales

1.8.2

go for multi-headed rmsnorm for the qknorm on hypersphere vit