pytorch-optimizer v3.4.2

Change Log

Feature

• Implement SCION optimizer. (#348, #352)
  • Training Deep Learning Models with Norm-Constrained LMOs

Update

• Update ScheduleFreeSGD, AdamW, RAdam optimizers with the latest. (#351, #353)
• Remove use_palm variant in ScheduleFree optimizer due to instability. (#353)
• Ranger25 optimizer. (#353)

Fix

• Remove weight decouple parameter in ScheduleFree optimizers. (#351, #353)

Docs

• Fix AliG optimizer visualization. (#350)

Contributions

thanks to @AidinHamedi, @hatonosuke