Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Nov 14, 2024

• support ZB-V in native pytorch

  Show description for 93af65c
  QPHutu

  authored
  93af65c

Commits on Nov 13, 2024

• Check transformer-impl for zero bubble (#51)

  huanggx-sea

  authored
  9b51cd7

• Add handcrafted_zb_v.py

  Show description for 106e470
  QPHutu

  authored
  106e470

Commits on Aug 19, 2024

• minor fix

  QPHutu

  authored
  6179cf2

Commits on Jul 4, 2024

• V schedules first implementation (#33)

  Show description for fd19c6b
  

  ufotalent

  and

  Wan Xinyi

  authored
  fd19c6b

Commits on Jun 28, 2024

• Fix optimizer rollback for distributed optimizer (#26)

  Show description for e7cf420
  

  QPHutu

  and

  QPH-SAIL

  authored
  e7cf420

• Support sequence-parallel for zero bubble schedule (#14)

  Show description for 331ee01
  

  QPHutu

  and

  QPH-SAIL

  authored
  331ee01

Commits on Jan 22, 2024

• Update paper link to arxiv

  ufotalent

  authored
  7ad9c81

Commits on Jan 17, 2024

• Add paper because arxiv pending too long

  Show description for 3a91d0c
  

  ufotalent

  and

  Wan Xinyi

  authored
  3a91d0c

Commits on Dec 27, 2023

• Add light weight and standalone ZB1P implementation and readme (#11)

  Show description for 7633c5f
  

  ufotalent

  and

  Wan Xinyi

  authored
  7633c5f

Commits on Dec 15, 2023

• Use custom memory settings for F/B/W on different stages (#6)

  Show description for 77b9eed
  

  3 peopleauthored
  77b9eed

Commits on Dec 7, 2023

• support distributed optimizer (#7)

  Show description for 25d191d
  

  QPHutu

  and

  QPH-SAIL

  authored
  25d191d

Commits on Dec 6, 2023

• Update README.md

  ufotalent

  authored
  f32308e

Commits on Nov 30, 2023

• Update README.md (#5)

  ufotalent

  authored
  73afc35

• minor fix on ZB-V (#3)

  Show description for 380d8af
  

  QPHutu

  and

  QPH-SAIL

  authored
  380d8af

Commits on Nov 29, 2023

• Merge pull request #2 from sail-sg/minor_change

  Show description for a33d5b1
  ufotalent

  authored
  a33d5b1

• minor change on auto schedule

  QPH-SAIL

  committed
  a914a97

• Merge pull request #1 from sail-sg/ufotalent-patch-1

  Show description for f65a145
  ufotalent

  authored
  f65a145

• Update README.md

  ufotalent

  authored
  d2b942d

Commits on Nov 28, 2023

• Don't generate precede matrix if ILP is not required

  ufotalent

  authored
  7cb8e1a

• Update README.md

  ufotalent

  authored
  e909e26

Commits on Nov 27, 2023

• Merge branch 'overlap_ar' into 'main'

  Show description for 5391c0f
  

  Xinyi Wan

  committed
  5391c0f

• Overlap all-reduce communication for DP

  

  QPH-SAIL

  authored and

  Xinyi Wan

  committed
  dac72f5

Commits on Nov 20, 2023

• Fix images in readme

  

  Xinyi Wan

  committed
  fbb3af2

• Add autodownload dataset for sample script

  

  Xinyi Wan

  committed
  7c89dd4

• Rename original README to Megatron.md

  

  Xinyi Wan

  committed
  c524d3a

• Functionality tests and ReadME

  

  Xinyi Wan

  committed
  425b42e

• Add tests for zero bubble

  

  Xinyi Wan

  committed
  a99d554

• Fix a minor bug for heuristic when m < p

  

  QPH-SAIL

  authored and

  Xinyi Wan

  committed
  b05105f

• disable optimizer post validation until grad norm < max norm

  

  QPH-SAIL

  authored and

  Xinyi Wan

  committed
  2d8cfdc

• support padding for num_layers

  

  QPH-SAIL

  authored and

  Xinyi Wan

  committed
  cff01bf

• + enable_zero_bubble, zero_bubble_interleaved -> zero_bubble_v_schedule

  

  QPH-SAIL

  authored and

  Xinyi Wan

  committed
  3763fbd

• Zero Bubble Pipeline Parallelism

  

  QPH-SAIL

  authored and

  Xinyi Wan

  committed
  ed85b11

Commits on Nov 18, 2023

• Merge branch 'memory_fixes' into 'main'

  Show description for 9290c73
  ericharper

  committed
  9290c73

• Clone output of view in _split_along_first_dim

  

  deepakn94

  authored and

  ericharper

  committed
  19afb90

Pagination

PreviousNext