Auto scaling factor tuning on FP8 weight gradients reduction for Megatron-LM#21
Open
wkcn wants to merge 3 commits intoAzure:mainfrom wkcn:wgrad_auto_scaling
+16-8
Commits
Commits on Dec 8, 2023
- committed
Commits on Dec 10, 2023
- committed
Commits on Dec 12, 2023
- committed