Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weight dropout #100

Closed
albertz opened this issue Feb 4, 2022 · 1 comment
Closed

Weight dropout #100

albertz opened this issue Feb 4, 2022 · 1 comment

Comments

@albertz
Copy link
Member

albertz commented Feb 4, 2022

Dropout on the weights.

This needs to depend (conditional logic, #24) on the train flag (#18). Both is implemented now, and we can just use:

with nn.Cond(nn.train_flag()):
  ...

Edit Actually not really because this is already internal to nn.dropout.

Also see nn.stochastic_depth as another example using nn.Cond with nn.train_flag.

Very related is also variational parameter noise (#240).

This was referenced Nov 11, 2022
@albertz
Copy link
Member Author

albertz commented Nov 12, 2022

This is implemented now, see nn.weight_dropout.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant