Code

Enable torch.autocast with ZeRO#6993

Code

Draft

tohtana wants to merge 6 commits intomasterdeepspeedai/DeepSpeed:masterfrom tohtana/support_autocastdeepspeedai/DeepSpeed:tohtana/support_autocastCopy head branch name to clipboard

+453-161

Commits

Commits on Jan 31, 2025

• add autocast support and ds_config item

  tohtana

  committed
  29c6bd0

  Copy full SHA for 29c6bd0

Commits on Feb 1, 2025

• prepare ipg buckets for multiple dtypes

  tohtana

  committed
  c5400cb

  Copy full SHA for c5400cb

Commits on Feb 2, 2025

• switch communication data type

  tohtana

  committed
  96573b6

  Copy full SHA for 96573b6

Commits on Feb 3, 2025

• add gradscaler

  tohtana

  committed
  0e52edb

  Copy full SHA for 0e52edb

• Merge branch 'master' into tohtana/support_autocast

  tohtana

  authored
  2202960

  Copy full SHA for 2202960

• fix import and formatting

  tohtana

  committed
  b294eae

  Copy full SHA for b294eae