-
Notifications
You must be signed in to change notification settings - Fork 180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG]: Kafka Message cannot be serialized to JSON when dict contains bytes #1621
Comments
I created a initial Draft on how this could be done in #1622, but this is not DRY. Once this has been vetted I would refactor the code and write the appropriate tests. |
I'm facing the same issue when my model outputs a numpy.array of string. It works fine when invoked through the REST API, but failed when invoked via Kafka server. I believe this is because of the difference on how I tried adding from fastapi.encoders import jsonable_encoder
def _encode_value(v: dict) -> bytes:
v = jsonable_encoder(v)
if orjson is None:
dumped = json.dumps(v)
return dumped.encode("utf-8")
return orjson.dumps(v) Using pydantic's encoder as the default encoder is also a possible solution (for from pydantic.json import pydantic_encoder
def _encode_value(v: dict) -> bytes:
if orjson is None:
dumped = json.dumps(v, default=pydantic_encoder)
return dumped.encode("utf-8")
return orjson.dumps(v, default=pydantic_encoder) |
@hansen-young we merged a contribution from @DerTiedemann #1622 that fixes this issue. Please submit a new ticket if there are things still that need to be fixed. |
Issue
When using the KafkaServer to process Inference Requests while using a model that produces results that contain raw bytes (e.g. any recent transformer), the response is unable to be written to the output topic.
The resulting error:
Context / How to reproduce
python --version
:Python 3.10.13
mlserver --version
:mlserver, version 1.4.0
requires
mlserver-huggingface
model-settings.json
:settings.json
:example.py
:Possible Solution
The http server has a special encoder that is used to make this possible, simplest idea would be to just reuse that.
The text was updated successfully, but these errors were encountered: