Skip to content

MoE v1.0.0

Latest
Latest
Compare
Choose a tag to compare
@RissyRan RissyRan released this 10 Sep 06:36
· 30 commits to main since this release
moe-v1.0.0
7631466

MoE v1.0.0 supports:

  • Megablox with Fully Sharded Data Parallelism (FSDP) and Token Parallelism (TP)
  • Dropping strategies with FSDP, TP, and Expert Parallelism (EP)
Assets 2
Loading