You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Your question
Ask a clear and concise question about Megatron-LM.
**
**
why the _p2p_ops func has the condition branches to distinguish between get_pipeline_model_parallel_rank() % 2 == 0 and get_pipeline_model_parallel_rank() % 2 != 0 ?
just for different send recv launch order?
but send next and recv prev will use different stream, no dependency
This discussion was converted from issue #865 on August 07, 2024 18:28.
Heading
Bold
Italic
Quote
Code
Link
Numbered list
Unordered list
Task list
Attach files
Mention
Reference
Menu
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Your question
Ask a clear and concise question about Megatron-LM.
**
**
why the _p2p_ops func has the condition branches to distinguish between get_pipeline_model_parallel_rank() % 2 == 0 and get_pipeline_model_parallel_rank() % 2 != 0 ?
just for different send recv launch order?
but send next and recv prev will use different stream, no dependency
Beta Was this translation helpful? Give feedback.
All reactions