Code

Feature/moe#55

Code

Merged

xrsrke merged 5 commits intomainxrsrke/pipegoose:mainfrom feature/moexrsrke/pipegoose:feature/moeCopy head branch name to clipboardDec 6, 2023

+549-160

Commits

Commits on Dec 2, 2023

• [Refactor] Expert WIP

  xrsrke

  committed
  12588c2

Commits on Dec 5, 2023

• [Feature] Add expert data parallel group

  xrsrke

  committed
  05562bb

• [Feature] Add support for data parallelism in MoE

  xrsrke

  committed
  4d7b344

• [Refactor] Refactor MoE with data parallelism's tests

  xrsrke

  committed
  e094986

Commits on Dec 6, 2023

• [Feature] Add support for tensor parallelism in MoE

  xrsrke

  committed
  dfda88c