Skip to content

v1.24 - The World's Highest Wingsuit Jump

Compare
Choose a tag to compare
@pzelasko pzelasko released this 05 Jun 19:59
· 21 commits to master since this release
4d57d53

What's Changed

New features

Notably, there's a new optimization for dynamic bucketing sampler in multi-GPU training - it will choose the same (or the closest possible) bucket on each DDP rank to keep the total training step times closer. The expected speedup is dependent on the model and the number of GPUs. We observed 8 and 13% speedups across two experiments compared to non-synchronized bucket selection. The new option is called sync_buckets and is enabled by default.

Recipes

Other improvements

New Contributors

Full Changelog: v1.23...v1.24