We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent cae5c3c commit b736ca9Copy full SHA for b736ca9
USER_GUIDE.md
@@ -16,7 +16,7 @@ export ZONE=your-tpu-zone
16
export PROJECT=your-tpu-project
17
```
18
19
-### HF Mixtral 7 x 8B Environment Setup
+### HF Llama Environment Setup
20
21
Here both PyTorch and PyTorch/XLA nightly are used with our fork of HuggingFace.
22
@@ -60,7 +60,7 @@ The last step for HF setup is to copy your fsdp_config.json into the TPU VM.
60
61
{
62
"fsdp_transformer_layer_cls_to_wrap": [
63
- "MixtralDecoderLayer"
+ "LlamaDecoderLayer"
64
],
65
"xla": true,
66
"xla_fsdp_v2": true,
0 commit comments