Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MoE #639

Open
wants to merge 144 commits into
base: main
Choose a base branch
from
Open

MoE #639

wants to merge 144 commits into from

Commits on Jun 20, 2024

  1. Clean MoE implementation

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    e725eb9 View commit details
    Browse the repository at this point in the history

  2. Add conf

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    db24750 View commit details
    Browse the repository at this point in the history

  3. Fix return args

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    18450de View commit details
    Browse the repository at this point in the history

  4. Rmv outdated kwarg

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    4ab7f77 View commit details
    Browse the repository at this point in the history

  5. Rmv legacy kwarg

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    dba42fd View commit details
    Browse the repository at this point in the history

  6. Merge branch 'Muennighoff/MoE' of github.com:allenai/LLM into Muennig… 

    …hoff/MoE
    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    6c5f8a3 View commit details
    Browse the repository at this point in the history

  7. Add distributed_strategy

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    6a8e089 View commit details
    Browse the repository at this point in the history

  8. Allow w/o weight attr

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    1a9a317 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'Muennighoff/MoE' of github.com:allenai/LLM into Muennig… 

    …hoff/MoE
    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    ddf6fd4 View commit details
    Browse the repository at this point in the history

  10. Allow w/o weight attr

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    ab55e07 View commit details
    Browse the repository at this point in the history

  11. Add MoE params

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    7aeefd4 View commit details
    Browse the repository at this point in the history

  12. Rmv kwarg

    @Muennighoff
    Muennighoff committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    3eab45c View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. Reduce lb & moe losses

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    6d736da View commit details
    Browse the repository at this point in the history

  2. LN & Emb Dec

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d07c638 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'Muennighoff/MoE' of github.com:allenai/LLM into Muennig… 

    …hoff/MoE
    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    cdb592f View commit details
    Browse the repository at this point in the history

  4. Do not decay emb

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    1399841 View commit details
    Browse the repository at this point in the history

  5. Tmp - debug throughput

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    a13b5b8 View commit details
    Browse the repository at this point in the history

  6. Fix

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    935167e View commit details
    Browse the repository at this point in the history

  7. Fix

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    b96972d View commit details
    Browse the repository at this point in the history

  8. maintain init order

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    0079490 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'Muennighoff/MoE' of github.com:allenai/LLM into Muennig… 

    …hoff/MoE
    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    8b1c441 View commit details
    Browse the repository at this point in the history

  10. Decay emb

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    e2c7286 View commit details
    Browse the repository at this point in the history

  11. Keep EA on CPU

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d39a37c View commit details
    Browse the repository at this point in the history

  12. Do not decay emb

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    3acfc04 View commit details
    Browse the repository at this point in the history

  13. Change norm

    @Muennighoff
    Muennighoff committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    4432261 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. Confs

    @Muennighoff
    Muennighoff committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    cef7707 View commit details
    Browse the repository at this point in the history

  2. Adapt wrap

    @Muennighoff
    Muennighoff committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    2a6df33 View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2024

  1. Add conf

    @Muennighoff
    Muennighoff committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    7421890 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. decemb conf

    @Muennighoff
    Muennighoff committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    021974e View commit details
    Browse the repository at this point in the history

  2. Updates

    @Muennighoff
    Muennighoff committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    d5a0626 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. up

    @Muennighoff
    Muennighoff committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    fce086f View commit details
    Browse the repository at this point in the history

  2. Update weka

    @Muennighoff
    Muennighoff committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    daa7c91 View commit details
    Browse the repository at this point in the history

  3. Rev weka

    @Muennighoff
    Muennighoff committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    3a40b7f View commit details
    Browse the repository at this point in the history

  4. Adapt

    @Muennighoff
    Muennighoff committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    2676d03 View commit details
    Browse the repository at this point in the history

  5. Add fine

    @Muennighoff
    Muennighoff committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    2fb4c96 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Adapt

    @Muennighoff
    Muennighoff committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    448a9a8 View commit details
    Browse the repository at this point in the history

  2. Adapt

    @Muennighoff
    Muennighoff committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    e361918 View commit details
    Browse the repository at this point in the history

  3. Add eps

    @Muennighoff
    Muennighoff committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    9377aa5 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Update confs

    @Muennighoff
    Muennighoff committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    bad3a34 View commit details
    Browse the repository at this point in the history

  2. Change eps

    @Muennighoff
    Muennighoff committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    3dbd156 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. Add conf

    @Muennighoff
    Muennighoff committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a6496d6 View commit details
    Browse the repository at this point in the history

  2. Confs

    @Muennighoff
    Muennighoff committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    5dd6135 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2024

  1. Add conf

    @Muennighoff
    Muennighoff committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    195c77d View commit details
    Browse the repository at this point in the history

  2. Fix path

    @Muennighoff
    Muennighoff committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    4ea9f0a View commit details
    Browse the repository at this point in the history

  3. Fix path

    @Muennighoff
    Muennighoff committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    44fa5ae View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2024

  1. Adapt confs

    @Muennighoff
    Muennighoff committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    b7658eb View commit details
    Browse the repository at this point in the history

  2. add s3 conf

    @Muennighoff
    Muennighoff committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    16edece View commit details
    Browse the repository at this point in the history

  3. fix paths

    @Muennighoff
    Muennighoff committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    f8e061f View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Update confs

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    c8a51cf View commit details
    Browse the repository at this point in the history

  2. Increase bs

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    b95a05f View commit details
    Browse the repository at this point in the history

  3. Adjust paths

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    20103ec View commit details
    Browse the repository at this point in the history

  4. conf

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    459ef27 View commit details
    Browse the repository at this point in the history

  5. confs

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    84f21db View commit details
    Browse the repository at this point in the history

  6. confs

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    08d3253 View commit details
    Browse the repository at this point in the history

  7. Update confs

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    4b07140 View commit details
    Browse the repository at this point in the history

  8. Update conf

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    cf6fa33 View commit details
    Browse the repository at this point in the history

  9. Conf

    @Muennighoff
    Muennighoff committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    d95f978 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. Finegrained

    @Muennighoff
    Muennighoff committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    3c15abf View commit details
    Browse the repository at this point in the history

  2. Cx5

    @Muennighoff
    Muennighoff committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    4163e70 View commit details
    Browse the repository at this point in the history

  3. Add conf

    @Muennighoff
    Muennighoff committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    aae0e0b View commit details
    Browse the repository at this point in the history

  4. Add cx5

    @Muennighoff
    Muennighoff committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    76e6e6d View commit details
    Browse the repository at this point in the history

  5. Up confs

    @Muennighoff
    Muennighoff committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    31e387b View commit details
    Browse the repository at this point in the history

  6. Up confs

    @Muennighoff
    Muennighoff committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    f631d8e View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. Dfix

    @Muennighoff
    Muennighoff committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    142720b View commit details
    Browse the repository at this point in the history

  2. reddit

    @Muennighoff
    Muennighoff committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    412a55e View commit details
    Browse the repository at this point in the history

  3. flan

    @Muennighoff
    Muennighoff committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    669bba5 View commit details
    Browse the repository at this point in the history

  4. fine

    @Muennighoff
    Muennighoff committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    7d90908 View commit details
    Browse the repository at this point in the history

  5. indent

    @Muennighoff
    Muennighoff committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3c97519 View commit details
    Browse the repository at this point in the history

  6. Add conf

    @Muennighoff
    Muennighoff committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    6e08b09 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. Conf

    @Muennighoff
    Muennighoff committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    1a6a2e2 View commit details
    Browse the repository at this point in the history

  2. Add shared

    @Muennighoff
    Muennighoff committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    103e450 View commit details
    Browse the repository at this point in the history

  3. Add

    @Muennighoff
    Muennighoff committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    48b6c14 View commit details
    Browse the repository at this point in the history

  4. Add conf

    @Muennighoff
    Muennighoff committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    8a387fa View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2024

  1. Add conf

    @Muennighoff
    Muennighoff committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    3203724 View commit details
    Browse the repository at this point in the history

  2. Add conf

    @Muennighoff
    Muennighoff committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    5309137 View commit details
    Browse the repository at this point in the history

  3. Addqk

    @Muennighoff
    Muennighoff committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    bf66e68 View commit details
    Browse the repository at this point in the history

  4. Make QK Norm parametric

    @Muennighoff
    Muennighoff committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    63c12e1 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2024

  1. dense comp

    @Muennighoff
    Muennighoff committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    1c4aa8d View commit details
    Browse the repository at this point in the history

  2. add conf

    @Muennighoff
    Muennighoff committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    6a94263 View commit details
    Browse the repository at this point in the history

  3. conf

    @Muennighoff
    Muennighoff committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    b58c316 View commit details
    Browse the repository at this point in the history

  4. conf

    @Muennighoff
    Muennighoff committed Jul 7, 2024
    Configuration menu
    Copy the full SHA
    3bbfaed View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. Change torch version

    @Muennighoff
    Muennighoff committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    364659f View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2024

  1. Add anneal

    @Muennighoff
    Muennighoff committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    15f5503 View commit details
    Browse the repository at this point in the history

  2. Resets

    @Muennighoff
    Muennighoff committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    6c516d8 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2024

  1. add conf

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    8a0758e View commit details
    Browse the repository at this point in the history

  2. conf

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    c23b048 View commit details
    Browse the repository at this point in the history

  3. Adjust mlp hidden

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    285ff10 View commit details
    Browse the repository at this point in the history

  4. Change ratio

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    99aec31 View commit details
    Browse the repository at this point in the history

  5. Add conf

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    a43eae8 View commit details
    Browse the repository at this point in the history

  6. Conf

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    aaefc58 View commit details
    Browse the repository at this point in the history

  7. Adjust

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    270271f View commit details
    Browse the repository at this point in the history

  8. add conf

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    c084d34 View commit details
    Browse the repository at this point in the history

  9. Fix conf

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    6789ee2 View commit details
    Browse the repository at this point in the history

  10. Adjust

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    f084fa0 View commit details
    Browse the repository at this point in the history

  11. Add alt

    @Muennighoff
    Muennighoff committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    14ee7e4 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Fix typo; update script

    @Muennighoff
    Muennighoff committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    f831adf View commit details
    Browse the repository at this point in the history

  2. merge main

    @Muennighoff
    Muennighoff committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    26eb3f3 View commit details
    Browse the repository at this point in the history

  3. add moe reorder

    @Muennighoff
    Muennighoff committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    0a3b076 View commit details
    Browse the repository at this point in the history

  4. Add datafix

    @Muennighoff
    Muennighoff committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    d6ccbf0 View commit details
    Browse the repository at this point in the history

  5. Max doc len MoE

    @Muennighoff
    Muennighoff committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    6a2c17e View commit details
    Browse the repository at this point in the history

  6. add fa varlen

    @Muennighoff
    Muennighoff committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    9ccb2f1 View commit details
    Browse the repository at this point in the history

  7. Add conf

    @Muennighoff
    Muennighoff committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    f5291ec View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. add conf

    @Muennighoff
    Muennighoff committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    ed571a8 View commit details
    Browse the repository at this point in the history

  2. fix conf

    @Muennighoff
    Muennighoff committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    71d2d2a View commit details
    Browse the repository at this point in the history

  3. fixconf

    @Muennighoff
    Muennighoff committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    8b72521 View commit details
    Browse the repository at this point in the history

  4. fixonf

    @Muennighoff
    Muennighoff committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    fc822a0 View commit details
    Browse the repository at this point in the history

  5. fix

    @Muennighoff
    Muennighoff committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    1ccaf9a View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Add 8k

    @Muennighoff
    Muennighoff committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    c157441 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. add conf

    @Muennighoff
    Muennighoff committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    06b8010 View commit details
    Browse the repository at this point in the history

  2. conf

    @Muennighoff
    Muennighoff committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    cf31e53 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. Adapt

    @Muennighoff
    Muennighoff committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    a750fde View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into Muennighoff/MoE

    @Muennighoff
    Muennighoff committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c853a43 View commit details
    Browse the repository at this point in the history

  3. Sort

    @Muennighoff
    Muennighoff committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    bea80ec View commit details
    Browse the repository at this point in the history

  4. Simplify configs for merge

    @Muennighoff
    Muennighoff committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    dfdcfc5 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. Update megablocks installation

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    4606598 View commit details
    Browse the repository at this point in the history

  2. Simplify

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    f6c707d View commit details
    Browse the repository at this point in the history

  3. Simplify

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    76f0376 View commit details
    Browse the repository at this point in the history

  4. typing & changes

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    9631c80 View commit details
    Browse the repository at this point in the history

  5. Format

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    2fa2acb View commit details
    Browse the repository at this point in the history

  6. Change torch.tensor to Any so type check passes?

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    c517703 View commit details
    Browse the repository at this point in the history

  7. Simplify

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    b4eb33f View commit details
    Browse the repository at this point in the history

  8. Simplify

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    ae6f16a View commit details
    Browse the repository at this point in the history

  9. Sort

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    02781be View commit details
    Browse the repository at this point in the history

  10. Fix type checks

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    a62a1ee View commit details
    Browse the repository at this point in the history

  11. Format

    @Muennighoff
    Muennighoff committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    0ecd4b8 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2024

  1. Fix typo; MoEArgs func

    @Muennighoff
    Muennighoff committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    d8452a0 View commit details
    Browse the repository at this point in the history

  2. Format

    @Muennighoff
    Muennighoff committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    8a28ced View commit details
    Browse the repository at this point in the history

  3. Check for act ckpt strategy & moe; fix typo

    @Muennighoff
    Muennighoff committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    91f5553 View commit details
    Browse the repository at this point in the history

  4. fix import

    @Muennighoff
    Muennighoff committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    61ac104 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Sort impot

    @Muennighoff
    Muennighoff authored Aug 20, 2024
    Configuration menu
    Copy the full SHA
    f4faf8a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into Muennighoff/MoE

    @Muennighoff
    Muennighoff authored Aug 20, 2024
    Configuration menu
    Copy the full SHA
    fdc1021 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. Fix typo 

    Co-authored-by: Shane A <[email protected]>
    @Muennighoff @2015aroras
    Muennighoff and 2015aroras authored Aug 21, 2024
    Configuration menu
    Copy the full SHA
    ed82181 View commit details
    Browse the repository at this point in the history

  2. Simplify isinstance 

    Co-authored-by: Shane A <[email protected]>
    @Muennighoff @2015aroras
    Muennighoff and 2015aroras authored Aug 21, 2024
    Configuration menu
    Copy the full SHA
    b0cc754 View commit details
    Browse the repository at this point in the history

  3. Clean conf & move constructor

    @Muennighoff
    Muennighoff committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    ca9b41f View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Add ref

    @Muennighoff
    Muennighoff committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    215c0f5 View commit details
    Browse the repository at this point in the history

  2. Merge main

    @Muennighoff
    Muennighoff committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    43baf74 View commit details
    Browse the repository at this point in the history

  3. Sort imports

    @Muennighoff
    Muennighoff committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    775e514 View commit details
    Browse the repository at this point in the history

  4. Format

    @Muennighoff
    Muennighoff committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    cd0004b View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. No exp ass

    @Muennighoff
    Muennighoff committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    acb23dd View commit details
    Browse the repository at this point in the history

  2. Revert

    @Muennighoff
    Muennighoff committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    a143469 View commit details
    Browse the repository at this point in the history

  3. Simplify

    @Muennighoff
    Muennighoff committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    1a4bdae View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. Rmv interleave

    @Muennighoff
    Muennighoff committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    410064a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into Muennighoff/MoE

    @Muennighoff
    Muennighoff committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    671bc8e View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. expert_assignments on gpu

    @Muennighoff
    Muennighoff committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    04a2da5 View commit details
    Browse the repository at this point in the history