Skip to content

Commit

Permalink
Update fsdp.md (#519)
Browse files Browse the repository at this point in the history
`torch.nn.Module.to_empty` takes keyword only arg of "device" according
to
https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.to_empty
  • Loading branch information
crcrpar authored Aug 14, 2024
1 parent 36a0057 commit 1c96a01
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/fsdp.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ fully_shard(model)
for tensor in itertools.chain(model.parameters(), model.buffers()):
assert tensor.device == torch.device("meta")
# Allocate buffers and sharded parameters on GPU
model.to_empty("cuda")
model.to_empty(device="cuda")
# Run user-defined initializers
model.init_weights() # or `model.apply(init_weights)`
```
Expand Down

0 comments on commit 1c96a01

Please sign in to comment.