Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issues with model serving #51

Open
erwangranger opened this issue Oct 7, 2024 · 2 comments
Open

issues with model serving #51

erwangranger opened this issue Oct 7, 2024 · 2 comments

Comments

@erwangranger
Copy link
Collaborator

I'm following:
https://docs.redhat.com/en/documentation/red_hat_openshift_ai_self-managed/2.13/html/openshift_ai_tutorial_-_fraud_detection_example/deploying-and-testing-a-model#deploying-a-model-single-model-server

in this, the screenshot is missing the route info:
image

So a user will be likely to not make anything external.

So their endpoint will look like:
image

And then, the notebook validation will fail.
In spite of allowed for self-signed certs in the code, it works well with the route, but not at all with the internal endpoint.

@cfchase
Copy link
Collaborator

cfchase commented Oct 11, 2024

Is there a bug filed against the OpenShift AI? In reproducing this, the behavior is different before and after creating a model server with an external route. If you create the internal service never having created an external service, it will fail with a connection reset. If you create the internal service after an external service exists, it works as expected. Leads me to believe that there's an issue with how the internal routing is set up in the project that the external routing initializes correctly.

@cfchase
Copy link
Collaborator

cfchase commented Oct 11, 2024

Another issue is that while the external endpoint had a verified ssl cert, the internal service does not. This seems like very strange and inconsistent behavior. It makes using these endpoints internally cumbersome even if it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants