Encoder-Decoder (T5) support (#605)

* feat: add t5 components (layernorm, attention, encoder\decoder block)

* fix: add a couple of fields to Config for T5

* feat: T5 class

* feat: add weight and config loading for T5

* fix: updated Config and deviced for t5

* feat: added tests for t5 model

* fix: some changes to T5attn and T5Block components to fix tests

* fix: fix EncoderDecoder details to fix tests

* fix: loading from pretrained T5

* fix: changed signarure of abstract attn add rel_pos bias support

* update docstr

* fix: isort

* fix(tests): increase tolerance for acceptance tests
to pass on github runners,  on my machine worked well

* fix(format): black

* fix(lint): black 2

* fix: mypy

* fix: format

* demo of T5

* fix: change asserts to ifs

* added demo to CI

* removed demo

* removed t5 from tests

---------

Co-authored-by: Bryce Meyer <[email protected]>

.github/workflows/checks.yml

@@ -133,6 +133,7 @@ jobs:
           - "Main_Demo"
           # - "No_Position_Experiment"
           - "Othello_GPT"
+          # - "T5"
     steps:
       - uses: actions/checkout@v3
       - name: Install Poetry