Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adding TP llama example #2623

Merged
merged 35 commits into from
Oct 5, 2023
Merged

adding TP llama example #2623

merged 35 commits into from
Oct 5, 2023

Commits on Sep 27, 2023

  1. adding TP llama example

    @HamidShojanazeri
    HamidShojanazeri committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    5858d1b View commit details
    Browse the repository at this point in the history

  2. clean up

    @HamidShojanazeri
    HamidShojanazeri committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    a46f570 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into llama-tp

    @HamidShojanazeri
    HamidShojanazeri authored Sep 27, 2023
    Configuration menu
    Copy the full SHA
    71c72ae View commit details
    Browse the repository at this point in the history

  4. adding check point converter

    @HamidShojanazeri
    HamidShojanazeri committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    c0a351f View commit details
    Browse the repository at this point in the history

  5. Merge branch 'llama-tp' of https://github.com/pytorch/serve into llam… 

    …a-tp
    @HamidShojanazeri
    HamidShojanazeri committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    32fe6a5 View commit details
    Browse the repository at this point in the history

  6. clean up

    @HamidShojanazeri
    HamidShojanazeri committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    3fabaf1 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2023

  1. addressingt the comments

    @HamidShojanazeri
    HamidShojanazeri committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    68b4657 View commit details
    Browse the repository at this point in the history

  2. fixing the error handling

    @HamidShojanazeri
    HamidShojanazeri committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    712b402 View commit details
    Browse the repository at this point in the history

  3. clean up

    @HamidShojanazeri
    HamidShojanazeri committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    2ad0d99 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2023

  1. fixing the batch issue for chat

    @HamidShojanazeri
    HamidShojanazeri committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    185a069 View commit details
    Browse the repository at this point in the history

  2. adding min max gpus for inference

    @HamidShojanazeri
    HamidShojanazeri committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    bd7d784 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into llama-tp

    @HamidShojanazeri
    HamidShojanazeri authored Sep 30, 2023
    Configuration menu
    Copy the full SHA
    fe7cd25 View commit details
    Browse the repository at this point in the history

  4. adding max seq len and batch size explanations

    @HamidShojanazeri
    HamidShojanazeri committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    8012f19 View commit details
    Browse the repository at this point in the history

  5. clean up

    @HamidShojanazeri
    HamidShojanazeri committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    415e38e View commit details
    Browse the repository at this point in the history

  6. lowering the new token gen number

    @HamidShojanazeri
    HamidShojanazeri committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    c5da92a View commit details
    Browse the repository at this point in the history

  7. clean up

    @HamidShojanazeri
    HamidShojanazeri committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    3f8790c View commit details
    Browse the repository at this point in the history

  8. Merge branch 'llama-tp' of https://github.com/pytorch/serve into llam… 

    …a-tp
    @HamidShojanazeri
    HamidShojanazeri committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    57ef9da View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2023

  1. fixing the hard coded tp_degree

    @HamidShojanazeri
    HamidShojanazeri committed Oct 1, 2023
    Configuration menu
    Copy the full SHA
    3baae39 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. adding flash attention v2

    @HamidShojanazeri
    HamidShojanazeri committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    01db2d9 View commit details
    Browse the repository at this point in the history

  2. adding note for tp size

    @HamidShojanazeri
    HamidShojanazeri committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    2af0a1c View commit details
    Browse the repository at this point in the history

  3. fixing spell checks

    @HamidShojanazeri
    HamidShojanazeri committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    ea33b4f View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into llama-tp

    @HamidShojanazeri
    HamidShojanazeri authored Oct 2, 2023
    Configuration menu
    Copy the full SHA
    8c11029 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. update packaging step

    @HamidShojanazeri
    HamidShojanazeri committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    ddd2437 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'llama-tp' of https://github.com/pytorch/serve into llam… 

    …a-tp
    @HamidShojanazeri
    HamidShojanazeri committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    683a4e5 View commit details
    Browse the repository at this point in the history

  3. update packaing step

    @HamidShojanazeri
    HamidShojanazeri committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    33ba693 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into llama-tp

    @HamidShojanazeri
    HamidShojanazeri authored Oct 3, 2023
    Configuration menu
    Copy the full SHA
    e2cade7 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. addressing comments on configs

    @HamidShojanazeri
    HamidShojanazeri authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    9d92f3b View commit details
    Browse the repository at this point in the history

  2. address comments on model config

    @HamidShojanazeri
    HamidShojanazeri authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    21c979a View commit details
    Browse the repository at this point in the history

  3. Update REAME.md

    @HamidShojanazeri
    HamidShojanazeri authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    324213e View commit details
    Browse the repository at this point in the history

  4. Update REAME.md

    @HamidShojanazeri
    HamidShojanazeri authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    c3f515a View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into llama-tp

    @HamidShojanazeri
    HamidShojanazeri authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    ebc3d7f View commit details
    Browse the repository at this point in the history

  6. Update REAME.md

    @HamidShojanazeri
    HamidShojanazeri authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    0fdccbd View commit details
    Browse the repository at this point in the history

  7. Merge branch 'master' into llama-tp

    @lxning
    lxning authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    730b931 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'master' into llama-tp

    @lxning
    lxning authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    2f37308 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2023

  1. Update REAME.md

    @HamidShojanazeri
    HamidShojanazeri authored Oct 5, 2023
    Configuration menu
    Copy the full SHA
    c507e11 View commit details
    Browse the repository at this point in the history