Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Transformer Engine support to Praxis #36

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

Add Transformer Engine support to Praxis #36

wants to merge 15 commits into from

Commits on Aug 26, 2024

  1. Adding TE support

    @mingxu1067 @terrykong
    mingxu1067 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    e46ee19 View commit details
    Browse the repository at this point in the history

  2. Fix missing vars wiht PP.

    @mingxu1067 @terrykong
    mingxu1067 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    7346aaf View commit details
    Browse the repository at this point in the history

  3. Add checkpoint_policy checker for fused attn + dropout 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    a6fb78e View commit details
    Browse the repository at this point in the history

  4. Support more TE configurations 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    9bf1d5a View commit details
    Browse the repository at this point in the history

  5. Change the gated activations orders 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    2b7dc68 View commit details
    Browse the repository at this point in the history

  6. Remove RoPE restriction from DPA module 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    ae4980c View commit details
    Browse the repository at this point in the history

  7. Add rotary_pos_emb_group dispatch 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    a92fa1c View commit details
    Browse the repository at this point in the history

  8. Fix the missing .cls 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    08be555 View commit details
    Browse the repository at this point in the history

  9. Fixed the unexpected input sharding pattern when TE enabled. 

    Signed-off-by: Ming-Xu Huang <[email protected]>
    @mingxu1067 @terrykong
    mingxu1067 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    be2bd3f View commit details
    Browse the repository at this point in the history

  10. Addind a comment to get_input_bld 

    Signed-off-by: Ming-Xu Huang <[email protected]>
    @mingxu1067 @terrykong
    mingxu1067 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    cd73548 View commit details
    Browse the repository at this point in the history

  11. Leverage the origianl input bld when ENABLE_SP=False 

    Signed-off-by: Ming Huang <[email protected]>
    @mingxu1067 @terrykong
    mingxu1067 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    74078ba View commit details
    Browse the repository at this point in the history

  12. Use causal_padding instead of padding 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    e22e36f View commit details
    Browse the repository at this point in the history

  13. Fix rope convert 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    3f23d15 View commit details
    Browse the repository at this point in the history

  14. Add test_te_helper.py 

    Signed-off-by: Reese Wang <[email protected]>
    @zlsh80826 @terrykong
    zlsh80826 authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    afc4dc0 View commit details
    Browse the repository at this point in the history

  15. Add LoRA support in TE (#11)

    @hemildesai @terrykong
    hemildesai authored and terrykong committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    29a8bba View commit details
    Browse the repository at this point in the history