You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Not a bug as much as an important note for MADE which are producing deltas on the input:
For the output node where all of its input connections masked, the bias for this neuron should also be masked and currently is not. The code inside MaskedLinear should be changed as follows:
def set_mask(self, mask): # called when the masks are created. passes in this mask.
mask = torch.from_numpy(mask.astype(np.uint8).T)
self.mask.data.copy_(mask)
# if all of the inputs are zero, need to ensure the bias
# is zeroed out!
self.bias_all_zero_mask = (mask.sum(dim=1)!=0).float()
def forward(self, input):
return F.linear(input, self.mask * self.weight, self.bias_all_zero_mask * self.bias)```
The text was updated successfully, but these errors were encountered:
Not a bug as much as an important note for MADE which are producing deltas on the input:
For the output node where all of its input connections masked, the bias for this neuron should also be masked and currently is not. The code inside MaskedLinear should be changed as follows:
The text was updated successfully, but these errors were encountered: