Skip to content

Commit

Permalink
optimizer state values should be taken as mean absolute otherwise you…
Browse files Browse the repository at this point in the history
… could underestimate changes
  • Loading branch information
grantbuster committed Nov 12, 2024
1 parent 06c5d99 commit 3570174
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion sup3r/models/abstract.py
Original file line number Diff line number Diff line change
Expand Up @@ -1071,7 +1071,8 @@ def get_optimizer_state(cls, optimizer):
state = {'learning_rate': lr}
for var in optimizer.variables:
name = var.name
var = var.numpy().flatten().mean() # collapse ndarrays
var = var.numpy().flatten()
var = np.abs(var).mean() # collapse ndarrays into mean absolute
state[name] = float(var)
return state

Expand Down

0 comments on commit 3570174

Please sign in to comment.