Update MoE examples #192
Merged
Update MoE examples #192
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review. |
dsikka
approved these changes
Sep 23, 2024
|
slightly unrelated but once your fp8 moe PR in vllm lands, we should add an e2e test case: https://github.com/vllm-project/llm-compressor/blob/main/tests/e2e/vLLM/test_vllm.py |
dbarbuzzi
added a commit
to dbarbuzzi/llm-compressor
that referenced
this pull request
Sep 23, 2024
mgoin
pushed a commit
that referenced
this pull request
Sep 27, 2024
* Add tests for examples * Ignore examples tests by default * Trailing comma * Add test for "quantizing_moe_fp8" example * Update "quantizing_moe" example tests * Add test for "compressed_inference" example folder * Remove unused import * Add new dependency * Different approach for "flash_attn" * Add comment about flash_attn requirement * Test additional quantizing_moe example script * Add decorator to skip based on available VRAM * Limit GPU usage in 'cpu_offloading' example * Add optional pytest-xdist parallelization * Reduce persistent /tmp usage * Fix parametrization in big_models * Add pytest mark for GPU count requirement * Add 'multi_gpu' pytest marker * Skip 'deepseek_moe_w4a16.py' by default * Fix skip mark * Mark 'ex_trl_distillation.py' as multi_gpu * Abstract command copy/run to helper functions * Update for MoE examples PR #192 * Reduce test marker/decorator redundancy * style fixes * Rip out unused run parallelization * Exclude 'deepseek_moe_w8a8_fp8' from multi-GPU * Use variable for repeated string literal * Use `requires_gpu_count` over `requires_gpu`
markmc
pushed a commit
to markmc/llm-compressor
that referenced
this pull request
Nov 13, 2024
Co-authored-by: dhuangnm <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update the FP8 Mixtral example to not use GPTQ and add it to examples list