Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PyTorch] Miscellaneous fixes for FA3 attention #1174

Merged
merged 29 commits into from
Oct 8, 2024
Merged

[PyTorch] Miscellaneous fixes for FA3 attention #1174

merged 29 commits into from
Oct 8, 2024

Commits on Sep 10, 2024

  1. add qkv descales to FA3 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    bcdc4d1 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    3ed49d0 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. fix sbhd shapes 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    1db61e2 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    6a86660 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into add_descales

    @cyanguwa
    cyanguwa authored Sep 17, 2024
    Configuration menu
    Copy the full SHA
    7da4b6c View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. Merge branch 'main' into add_descales

    @cyanguwa
    cyanguwa authored Sep 18, 2024
    Configuration menu
    Copy the full SHA
    de3db0a View commit details
    Browse the repository at this point in the history

  2. force the same dtype when comparing FA3 and cuDNN FP8 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    19e7f87 View commit details
    Browse the repository at this point in the history

  3. Revert "force the same dtype when comparing FA3 and cuDNN FP8" 

    This reverts commit 19e7f87.

Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    bff80b6 View commit details
    Browse the repository at this point in the history

  4. force the same dtype when comparing FA3 and cuDNN FP8 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    68b9b48 View commit details
    Browse the repository at this point in the history

  5. add try/except for FA3 when custom qkv descales are not supported 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    0553a83 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. replace FA3 installation warning with a debug logging message 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    b73760b View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    66cc6f2 View commit details
    Browse the repository at this point in the history

  3. fix lint 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    3269685 View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    39a4e1d View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into add_descales 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa authored Sep 19, 2024
    Configuration menu
    Copy the full SHA
    5bcc355 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Merge branch 'main' into add_descales

    @cyanguwa
    cyanguwa authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    3dbee25 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. Merge branch 'NVIDIA:main' into add_descales

    @cyanguwa
    cyanguwa authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    c01a5b2 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. Merge branch 'main' into add_descales

    @cyanguwa
    cyanguwa authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    12dc8a9 View commit details
    Browse the repository at this point in the history

  2. remove unused imports 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    336a452 View commit details
    Browse the repository at this point in the history

  3. avoid varlen_func for FP8 and improve messaging 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    2e140c5 View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    1138edf View commit details
    Browse the repository at this point in the history

  5. add SWA support for FA3 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    4095be8 View commit details
    Browse the repository at this point in the history

  6. fix lint 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    8e2bcc2 View commit details
    Browse the repository at this point in the history

  7. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    7bf4936 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2024

  1. change preference reason for FP8 logic 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    b765f3d View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. minor fixes 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    a4030e8 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    569532a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into add_descales

    @cyanguwa
    cyanguwa authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    e907ad7 View commit details
    Browse the repository at this point in the history

  4. minor fix 

    Signed-off-by: Charlene Yang <[email protected]>
    @cyanguwa
    cyanguwa committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    f006a25 View commit details
    Browse the repository at this point in the history