Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PyTorch] FP8 MHA with RoPE and Miscellaneous Improvements #1100

Merged
merged 24 commits into from
Sep 5, 2024
Merged

[PyTorch] FP8 MHA with RoPE and Miscellaneous Improvements #1100

merged 24 commits into from
Sep 5, 2024

Commits on Aug 15, 2024

  1. fp8 mha with rope 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    59b99ca View commit details
    Browse the repository at this point in the history

  2. avoid index select in cast ops 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    c46f82c View commit details
    Browse the repository at this point in the history

  3. avoid index select in fused_attn_fwd 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    dafd73f View commit details
    Browse the repository at this point in the history

  4. rename is_first_module_in_mha to fp8_output 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    0d2ff34 View commit details
    Browse the repository at this point in the history

  5. resolve comments 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    0e837c3 View commit details
    Browse the repository at this point in the history

  6. resolve comments 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    33c3ed6 View commit details
    Browse the repository at this point in the history

  7. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    13feabb View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. move transpose to backward for fp8 input 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    ae856e4 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. fix ut 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    7e26d22 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. Merge branch 'main' into xiny/fp8_mha_with_rope

    @yaox12
    yaox12 committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    fae44b6 View commit details
    Browse the repository at this point in the history

  2. resolve comments 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    521c77a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into xiny/fp8_mha_with_rope

    @yaox12
    yaox12 committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    10c6961 View commit details
    Browse the repository at this point in the history

  4. update argument list for CP 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    dd30c2d View commit details
    Browse the repository at this point in the history

  5. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    a94b3ad View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. Merge branch 'main' into xiny/fp8_mha_with_rope

    @yaox12
    yaox12 committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    bf56399 View commit details
    Browse the repository at this point in the history

  2. fix for FA3 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    400d526 View commit details
    Browse the repository at this point in the history

  3. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    b935e13 View commit details
    Browse the repository at this point in the history

  4. remove unnecessary copy of scale_inv 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    9eca369 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. skip fp8 dpa/mha tests when fa3 is not available 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    e3b75db View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. Merge branch 'main' into xiny/fp8_mha_with_rope

    @yaox12
    yaox12 committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    46d428f View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. Merge branch 'main' into xiny/fp8_mha_with_rope

    @yaox12
    yaox12 committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    6b80dd6 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Merge branch 'main' into xiny/fp8_mha_with_rope

    @yaox12
    yaox12 committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    df6132f View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Merge branch 'main' into xiny/fp8_mha_with_rope

    @yaox12
    yaox12 committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    c017154 View commit details
    Browse the repository at this point in the history

  2. fix a merge bug 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    f9da6d7 View commit details
    Browse the repository at this point in the history