Skip to content

Commit

Permalink
Disable ninja by default (microsoft#5194)
Browse files Browse the repository at this point in the history
microsoft#5192 reports an issue with the latest DeepSpeed release (0.13.3)
related to pre-compilation and the recently re-enabled `ninja` support
in microsoft#5088. Reverting to disabling `ninja` by default, but users can still
enable it with `DS_ENABLE_NINJA=1` until we can further debug to
understand the problem.
  • Loading branch information
mrwyattii authored and SNahir committed Mar 11, 2024
1 parent d69de8f commit 0af0e95
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/nv-pre-compile-ops.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
#python -c "import torch; print('CUDA available:', torch.cuda.is_available())"
- name: Compile DeepSpeed Ops
run: |
TORCH_CUDA_ARCH_LIST="7.0;7.5;8.0" DS_BUILD_OPS=1 DS_BUILD_SPARSE_ATTN=0 DS_BUILD_CUTLASS_OPS=0 DS_BUILD_RAGGED_DEVICE_OPS=0 DS_BUILD_EVOFORMER_ATTN=0 pip3 install .
DS_ENABLE_NINJA=1 TORCH_CUDA_ARCH_LIST="7.0;7.5;8.0" DS_BUILD_OPS=1 DS_BUILD_SPARSE_ATTN=0 DS_BUILD_CUTLASS_OPS=0 DS_BUILD_RAGGED_DEVICE_OPS=0 DS_BUILD_EVOFORMER_ATTN=0 pip3 install .
- name: DS Report
run: |
ds_report
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ def get_env_if_set(key, default: typing.Any = ""):
# For any pre-installed ops force disable ninja.
if torch_available:
from accelerator import get_accelerator
use_ninja = not is_env_set("DS_DISABLE_NINJA")
use_ninja = is_env_set("DS_ENABLE_NINJA")
cmdclass['build_ext'] = get_accelerator().build_extension().with_options(use_ninja=use_ninja)

if torch_available:
Expand Down

0 comments on commit 0af0e95

Please sign in to comment.