Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements cosine decay with linear warmup #15

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

YgorCastor
Copy link

This pull request introduces a new cosine decay schedule with linear warmup to the Polaris.Schedules module and adds corresponding tests to ensure its functionality.

New Feature: Cosine Decay Schedule with Linear Warmup

  • lib/polaris/schedules.ex: Added the warmup_cosine_decay function, which implements a learning rate schedule that first increases linearly during a warmup period and then follows a cosine decay pattern. This function includes detailed documentation and options for customization.

Tests for New Feature

  • test/polaris/schedules_test.exs: Added a series of tests for the warmup_cosine_decay function to verify its behavior, including tests for correct function arity, linear warmup phase, cosine decay phase, and respect for the alpha parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant