v0.5.0 Alpha 0 - The FSDP Release

The FSDP Release

This release of the training library introduces FSDP support in addition to the existing DeepSpeed support through the accelerate library.

What's Changed

• fix lint error to make CI happy by @RobotSail in #173
• Add license identifiers by @danmcp in #157
• Fix typos by @danmcp in #158
• feat: add e2e test for instructlab CI by @RobotSail in #174
• Ap/fix multipack for non granite models by @aldopareja in #169
• feat: add mergify by @RobotSail in #187
• ci: update runner labels to uniquely identify instance sizes by @nathan-weinberg in #180
• ci: minor cleanup of E2E job by @nathan-weinberg in #189
• Fixing e2e to use relative path for working-directory by @danmcp in #191
• Fix linting error and missing quote by @danmcp in #192
• Fix typos by @danmcp in #190
• Bump step-security/harden-runner from 2.8.1 to 2.9.1 by @dependabot in #159
• ci: add AWS tags to show github ref and PR num for all jobs by @nathan-weinberg in #196
• Bump pypa/gh-action-pypi-publish from 1.9.0 to 1.10.2 by @dependabot in #219
• Adding FSDP Support to Training Library by @aldopareja @Maxusmusti @RobotSail in #213
• Bump actions/setup-python from 5.1.0 to 5.2.0 by @dependabot in #193
• Bump rhysd/actionlint from 1.7.1 to 1.7.2 in /.github/workflows by @dependabot in #221
• ci: fix lint action by @nathan-weinberg in #227
• Bump hynek/build-and-inspect-python-package from 2.6.0 to 2.9.0 by @dependabot in #198
• Bump DavidAnson/markdownlint-cli2-action from 16.0.0 to 17.0.0 by @dependabot in #199

New Contributors

• @danmcp made their first contribution in #157

Full Changelog: v0.4.2...v0.5.0a0

v0.4.2

What's Changed

• Provide safeguards during training by @RobotSail in #168

Full Changelog: v0.4.1...v0.4.2

v0.4.1

What's Changed

• makes saving every save_samples an optional feature by @JamesKunstle in #165

Full Changelog: v0.4.0...v0.4.1

v0.4.0

What's Changed

• Change success message at end of training by @alinaryan in #156
• Adds a flag to save checkpoints at the end of an epoch by @JamesKunstle in #160

New Contributors

• @alinaryan made their first contribution in #156

Full Changelog: v0.3.2...v0.4.0

v0.3.2

What's Changed

• patch some hyper parameter arg descriptions in README by @jaideepr97 in #161
• Accept tuples for lora.target_modules by @booxter in #162

New Contributors

• @booxter made their first contribution in #162

Full Changelog: v0.3.1...v0.3.2

v0.3.1

What's Changed

• Update requirements to have bitsandbytes min and dolomite min by @Maxusmusti in #153

Full Changelog: v0.3.0...v0.3.1

v0.3.0

What's Changed

• remove dead code by @RobotSail in #138
• Updating token masking to support pretraining w/ masked special tokens by @Maxusmusti in #137
• fix: changes the check to check against both the enum option and enum value. Currently when we compare the type that gets passed in from the training args by only doing a check against the enum by @RobotSail in #143
• Adding weight merging for LoRA/QLoRA ckpts by @Maxusmusti in #142

Full Changelog: v0.2.0...v0.3.0

v0.2.0

What's Changed

• Bump actions/download-artifact from 4.1.7 to 4.1.8 by @dependabot in #127
• Fix ckpt save to include architecture for inference runtime consumption by @Maxusmusti in #129
• Reducing deepspeed timeout to 10mins by @Maxusmusti in #132
• Logging updates by @Maxusmusti in #134

Full Changelog: v0.1.0...v0.2.0

v0.1.0

What's Changed

• Reduce Unnecessary Multiprocessing by @Maxusmusti in #79
• Flash Attention Disable Toggle (Take 2) by @Maxusmusti in #118
• 🐛: fix optimizer selection logic so that FusedAdam is never loaded when CPU offloading is enabled by @RobotSail in #122
• Add wheel to requirements by @Maxusmusti in #123

Full Changelog: v0.0.5.1...v0.1.0

Updates Lazy Import Logic

This release includes PR #121 to overcome an issue where our way of lazily importing the run_training function is being picked up as an error by pylint.