Skip to content

Add a new mode to module so accGradInput will override the gradient instead of accumulate #1524

Open
@yiheng

Description

@yiheng

Module has an accGradInput method, which accumulates grad into a tensor. So we need to zero that tensor at the beginning of each iteration.

This is not necessary if we only accumulate once in each iteration. We can provide a mode flag. If the user set that flag, the accumulating gradient will become overwrite the tensor, which can save the zeroGrad time in the training.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions