Skip to content

Commit

Permalink
Upgrade neuron to 2.20.2
Browse files Browse the repository at this point in the history
This change also fixed loading Neuron models. Neuron uses old pytorch,
 before the change I was getting

TypeError: infer_schema() takes 1 positional argument but 2 were given

which was caused by custom_op direct registration. I'm disabling that
if custom_op is not available.
  • Loading branch information
Jerzy Zagorski committed Dec 9, 2024
1 parent d1c2e15 commit 991c004
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 2 deletions.
3 changes: 2 additions & 1 deletion Dockerfile.neuron
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# default base image
ARG BASE_IMAGE="public.ecr.aws/neuron/pytorch-inference-neuronx:2.1.2-neuronx-py310-sdk2.20.0-ubuntu20.04"
# https://gallery.ecr.aws/neuron/pytorch-inference-neuronx
ARG BASE_IMAGE="public.ecr.aws/neuron/pytorch-inference-neuronx:2.1.2-neuronx-py310-sdk2.20.2-ubuntu20.04"

FROM $BASE_IMAGE

Expand Down
2 changes: 1 addition & 1 deletion vllm/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -1628,7 +1628,7 @@ def direct_register_custom_op(
library object. If you want to bind the operator to a different library,
make sure the library object is alive when the operator is used.
"""
if is_in_doc_build():
if is_in_doc_build() or not supports_custom_op():
return
import torch.library
if hasattr(torch.library, "infer_schema"):
Expand Down

0 comments on commit 991c004

Please sign in to comment.