Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sampling callback #23

Merged
merged 6 commits into from
Apr 17, 2024
Merged

Sampling callback #23

merged 6 commits into from
Apr 17, 2024

Conversation

rousseab
Copy link
Collaborator

A first crack at periodically sampling the model while training, through a callback.

I also refactor how the loss is logged, to make it super extra explicit what is logged at every step and what is aggregated at every epoch.

There's a lot of possible improvement as future work:

  • there's not a lot of repetition in the input parameters... It would be nice to not have to repeat ourselves
  • the histogram image that is logged is pretty bare boned: it would be nice to overlay over the validation data, but that's complicated to do right now.
Screenshot 2024-04-16 at 4 06 28 PM

@rousseab rousseab requested a review from sblackburn86 April 16, 2024 20:07
@rousseab rousseab merged commit f6694aa into main Apr 17, 2024
1 check passed
@rousseab rousseab deleted the sampling_callback branch April 17, 2024 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants