How to convert pytorch model to onnx to use ragged batching? #5873
Unanswered
Sitcebelly
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I prepeared simple example.
I created simple summing model which have input and length in shape = [-1]
And triton adopted this model.
With config like:
But when i try to run this model with tritonhttpclient it throw me error
InferenceServerException: [400] onnx runtime error 2: Unexpected input data type. Actual: (tensor(float)) , expected: (tensor(int64))
It's also interesting why in example in documentation datatype for BATCH_ACCUMULATED_ELEMENT_COUNT is float32 not int.
Ok, after that i change input type to int64 for lengths in my config to int64 and now it not put it to triton, only put in with dtype TYPE_INT32, but torch.tensor([2, 7, 10]) have shape int64.
What i do wrong? Could you help me please?
Beta Was this translation helpful? Give feedback.
All reactions