Skip to content

Commit

Permalink
minor fix
Browse files Browse the repository at this point in the history
  • Loading branch information
QPHutu authored Aug 19, 2024
1 parent fd19c6b commit 6179cf2
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion megatron/core/weight_grad_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def clear(cls, model, chunk=0):
weight_grad_tasks[i] = None # release memory

# timers('wait_all_reduce', log_level=1).start(barrier=False)
for handle in embedding_handles:
for handle in handles:
if handle is not None:
handle.wait()
# timers('wait_all_reduce').stop()

0 comments on commit 6179cf2

Please sign in to comment.