Discussions

• 1 You must be logged in to vote

  🙏

  [QUESTION]Where does the attention_mask come from when the gpt_model is not the first or last pipeline stage?

  janelu9 asked Jun 8, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  [BUG] selective activation recompute only decrease little of GPU memory usage during training

  bugm asked Oct 18, 2024 in Q&A · Answered

  

  

  

  

  11
• 1 You must be logged in to vote

  💬

  Selecting Which layers to recompute activations in?

  eliird started Dec 22, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  [QUESTION]I cannot figure out TE(transformer_engine)

  Guodanding asked Aug 29, 2024 in Q&A · Unanswered

  

  

  

  

  15
• 1 You must be logged in to vote

  🙏

  [QUESTION] Training Llama3 70B on 16 x A100 only achieves low throughput of 20 TFLOPS

  ZeroAGI asked Aug 13, 2024 in Q&A · Unanswered

  

  

  

  

  4
• 3 You must be logged in to vote

  💡

  [ENHANCEMENT] Integrating torch.compile with Megatron/TransformerEngine

  yanboliang started Aug 20, 2024 in Ideas

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  [QUESTION]Using FP8 OOM, otherwise --bf16 works well

  yanchenmochen asked Oct 23, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Does Megatron support tracing computation graphs with torch.fx?

  fy-j asked Dec 15, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  What git branch corresponds to the latest version and what Docker version to use?

  karim-manaouil asked Dec 4, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  [QUESTION] How to load checkpoint saved in one parallel configuration (tensor/pipeline/data parallelism) can be loaded in a different parallel configuration ?

  polisettyvarma asked Oct 21, 2024 in Q&A · Unanswered

  

  

  3
• 3 You must be logged in to vote

  🙏

  [QUESTION]　To convert a Llama 3.1 70B checkpoint in torch dcp format to the HuggingFace format,

  kaiyama12345679 asked Oct 19, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  How is micro-batch-size influencing the throughput per GPU ?

  bugm asked Oct 12, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  [QUESTION] How Do NCCL_ALGO and Flash Attention Affect Deterministic Training in Megatron?

  jinzhuer asked Jul 11, 2024 in Q&A · Unanswered

  

  

  

  

  2
• 5 You must be logged in to vote

  🙏

  When will megatron Flash attention 3 be supported？

  echo-valor asked Jul 19, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Megatron FP8 training is compatible with recompute?

  yanchenmochen asked Oct 23, 2024 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  [QUESTION] Do tp overlap support thd, whose sequence length is flexible?

  wplf asked Oct 23, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  [QUESTION]How to setup fp8 trainging using Megatron-LM

  yanchenmochen asked Oct 11, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  [QUESTION] What's the internal difference for training when setting only "fp8-format" or setting "fp8-format"+"bf16"

  stale No activity in 60 days on issue or PR

  dong-liuliu asked Jun 21, 2024 in Q&A · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  [QUESTION]how to incorporate MOE into hybrid mamba efficiently

  sunying2018 asked Oct 21, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  [QUESTION] When will multimodal evaluation support pipeline parallelism ?

  wangxiaoyang-dev asked Oct 21, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  [QUESTION] why all rank0 of tp group make datasets？

  ltm920716 asked Oct 18, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  [QUESTION] How will recompute-num-layers influence the gpu memory usage with uniform recompute-method.

  bugm asked Oct 18, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  [QUESTION] Error in pipeline parallelism training for multimodal model

  CFC87 asked Oct 15, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  how to use TikTokenizer during Training?

  bugm asked Oct 12, 2024 in Q&A · Unanswered

  

  

  0
• 4 You must be logged in to vote

  🙏

  [QUESTION] How to convert a checkpoint to virtual pipeline format checkpoint

  ZacWang asked Oct 11, 2024 in Q&A · Unanswered

  

  

  0