Merge with upstream #48
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…PE tests to use fusion Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: Xin Yao <[email protected]>
Correct strides for bshd layout and revert RoPE tests to use fusion See merge request ADLR/megatron-lm!2469
Co-authored-by: Boxin Wang <[email protected]>
Add model checkpoint links See merge request ADLR/megatron-lm!2494
…ks that don't have trainable params Co-authored-by: Jon Barker <[email protected]>
Support --freeze-LM and --freeze-ViT with ranks that don't have trainable params See merge request ADLR/megatron-lm!2285
Move mmodal evaluation code to its own folder See merge request ADLR/megatron-lm!2491
Co-authored-by: Huy Vu2 <[email protected]>
Updating T5 codes to fix bugs See merge request ADLR/megatron-lm!2471
ci: Add memory consumption to tests See merge request ADLR/megatron-lm!2467
…m norm in a memory-efficient way
Reuse optimizer's main_params to compute param norm in a memory-efficient way See merge request ADLR/megatron-lm!2483
Co-authored-by: Oliver Koenig <[email protected]>
Add NeMo MoE test. See merge request ADLR/megatron-lm!2460
ci: Move most of LTS tests to nightly See merge request ADLR/megatron-lm!2496
Video training See merge request ADLR/megatron-lm!2500
ci: Update golden values of nightlies See merge request ADLR/megatron-lm!2511
…r newly added requests
Make generate function only return results for newly added requests See merge request ADLR/megatron-lm!2370
ci: Use torchrun See merge request ADLR/megatron-lm!2507
chore: Fix local generator script See merge request ADLR/megatron-lm!2519
Co-authored-by: William Dykas <[email protected]> Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: William Dykas <[email protected]> Co-authored-by: root <[email protected]>
[dist ckpt] Remove alias LocalNonpersitentObject See merge request ADLR/megatron-lm!2458
…and_type` during saving
[dist ckpt] Resolve todos in `_split_by_size_and_type` during saving See merge request ADLR/megatron-lm!2623
Co-authored-by: Mcore Bot <[email protected]>
Embedder fix for radio See merge request ADLR/megatron-lm!2651
tests: test_builder See merge request ADLR/megatron-lm!2670
Llava unit test fix See merge request ADLR/megatron-lm!2669
…ed with local checkpointing.
Warn instead of error when model_opt is enabled with local checkpointing. See merge request ADLR/megatron-lm!2667
Reduce NCCL memory cost in UT See merge request ADLR/megatron-lm!2633
Enabling UCC backend for PP communication See merge request ADLR/megatron-lm!2116
Fix for Frozen QK LayerNorm when training VLM with freeze VIT See merge request ADLR/megatron-lm!2632
chore: Bump version See merge request ADLR/megatron-lm!2680
Basic context and sequence parallel support in VLM example See merge request ADLR/megatron-lm!2561
Co-authored-by: jianbinc <[email protected]> Co-authored-by: 顾慎 <[email protected]> Co-authored-by: 李鹏 <[email protected]> Co-authored-by: 黄俊 <[email protected]> Co-authored-by: lostkevin <[email protected]> Co-authored-by: lostkevin <[email protected]> Co-authored-by: root <[email protected]>
Optimizer CPU offload support See merge request ADLR/megatron-lm!2526
Fix distributed checkpoint tests See merge request ADLR/megatron-lm!2675
Co-authored-by: Slawomir Kierat <[email protected]> Co-authored-by: Helen Ngo <[email protected]> Co-authored-by: Mcore Bot <[email protected]>
Statically allocate KV cache for MCore inference See merge request ADLR/megatron-lm!2641
Co-authored-by: Deepak Narayanan <[email protected]> Co-authored-by: Cyril Meurillon <[email protected]>
Various improvements to RerunStateMachine See merge request ADLR/megatron-lm!2659
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.