Skip to content

Commit

Permalink
[hotfix] fix zero comm buffer init (#6154)
Browse files Browse the repository at this point in the history
  • Loading branch information
ver217 authored Dec 10, 2024
1 parent 8d826a3 commit de3d371
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion colossalai/zero/low_level/low_level_optim.py
Original file line number Diff line number Diff line change
Expand Up @@ -371,7 +371,7 @@ def _run_reduction(self):
for i, sz in enumerate(bucket_store.sizes):
grp = bucket_store.torch_pg if len(bucket_store.sizes) == 1 else bucket_store.torch_pg[i]
flat_grads_list = list(cur_flat_grads.split(len(cur_flat_grads) // sz))
received_grad = torch.zeros_like(flat_grads_list[0])
received_grad = torch.empty_like(flat_grads_list[0])
if self._fp8_communication:
reduce_scatter_fp8(
received_grad,
Expand Down

0 comments on commit de3d371

Please sign in to comment.