Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flash attention support softcap. #1013

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Flash attention support softcap. #1013

wants to merge 11 commits into from

Commits on Jul 13, 2024

  1. support softcap for flash-attn

    @Lzhang-hub
    Lzhang-hub committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    34adf28 View commit details
    Browse the repository at this point in the history

  2. update flash-attn max version

    @Lzhang-hub
    Lzhang-hub committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    f6cd21c View commit details
    Browse the repository at this point in the history

  3. delete noused code

    @Lzhang-hub
    Lzhang-hub committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    7050f7b View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. update flash attn verion in setup

    @Lzhang-hub
    Lzhang-hub committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    36face8 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. no log attention backend

    @Lzhang-hub
    Lzhang-hub committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    cfbc218 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    cfe16c3 View commit details
    Browse the repository at this point in the history

  3. back

    @Lzhang-hub
    Lzhang-hub committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    57d05ac View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of https://github.com/Lzhang-hub/TransformerEngine

    @Lzhang-hub
    Lzhang-hub committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    5b75edc View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. check flash-attn for softcap 

    Signed-off-by: zhanglei335 <[email protected]>
    @Lzhang-hub
    Lzhang-hub committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    61c5a94 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. Merge branch 'main' into main

    @Lzhang-hub
    Lzhang-hub committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    28df6cc View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. softcap added to fa_optional_forward_kwargs 

    Signed-off-by: zhanglei335 <[email protected]>
    @Lzhang-hub
    Lzhang-hub committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    7851ab3 View commit details
    Browse the repository at this point in the history