Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PyTorch] Normalization ops #1033

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open

[PyTorch] Normalization ops #1033

wants to merge 31 commits into from

Commits on Jul 20, 2024

  1. Add layer norm op 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    338e193 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Add FP8 cast op 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    84bc1d7 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    0c40c54 View commit details
    Browse the repository at this point in the history

  3. Add tests for linear and layernorm with FP8 output 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    a7f0228 View commit details
    Browse the repository at this point in the history

  4. RMSNorm op 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    cb9c455 View commit details
    Browse the repository at this point in the history

  5. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    68635ad View commit details
    Browse the repository at this point in the history

  6. Fix linter warnings 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    cb9b4ec View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    b33f367 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    d9fb6f4 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. Replace LayerNorm module with LayerNorm op 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    00592d7 View commit details
    Browse the repository at this point in the history

  2. Replace RMSNorm module with RMSNorm op 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    e0a2fd9 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Add AMP support 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    ad32d6a View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    92d1f89 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    2197bca View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    c27a783 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. Merge branch 'main' into norm-ops 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    fb6b7e4 View commit details
    Browse the repository at this point in the history

  2. Do not save autograd context if grad mode is disabled 

    Debugging ONNX export tests.

Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    7be0524 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    e7c9c67 View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    91e6a03 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    21086aa View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. Forward args in pre_forward func to base op class 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    28bc058 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    e6c5d5f View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    4fdc3b7 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. Merge branch 'main' into norm-ops 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    b1141f5 View commit details
    Browse the repository at this point in the history

  2. Update to use QuantizedTensor class 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    4206fa2 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. Apply suggestions from code review 

    Co-authored-by: Przemyslaw Tredak <[email protected]>
Signed-off-by: Tim Moon <[email protected]>
    @timmoon10 @ptrendx
    timmoon10 and ptrendx committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    fd5afe5 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    5b90e4b View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Merge branch 'main' into norm-ops

    @timmoon10
    timmoon10 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    fd4ef97 View commit details
    Browse the repository at this point in the history

  2. Review suggestions from @ptrendx 

    Rename "CastFloat8" op to "Quantize". Add more fine-grained control for SM margin. Add docs for legacy sequence_parallel kwarg.

Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    102c64f View commit details
    Browse the repository at this point in the history

  3. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    87ce450 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Fix linter warnings 

    Signed-off-by: Tim Moon <[email protected]>
    @timmoon10
    timmoon10 committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    fed61f9 View commit details
    Browse the repository at this point in the history