Missing docker image for elyra/enterprise-gateway:3.2.2 on Dockerhub #1287

Chiragasourabh · 2023-04-06T16:23:30Z

Chiragasourabh
Apr 6, 2023

I see that there is a new release of [enterprise-gateway v3.2.2] (https://github.com/jupyter-server/enterprise_gateway/releases/tag/v3.2.2) but the respective docker image for the release is not available in the docker hub yet. Is there any specific reason for not publishing the docker image? if not, when could an image for the release can be expected on the docker hub?

Thank you

Answered by lresende

Apr 10, 2023

Done, the kerne-tf-gpu-py for now is a problem to publish multi-arch, so I just publish x86.

View full answer

anujpb · 2023-04-07T02:56:45Z

anujpb
Apr 7, 2023

@kevin-bates Do you know if there was a reason why the docker image for enterprise gateway was not published?

0 replies

kevin-bates · 2023-04-07T14:14:19Z

kevin-bates
Apr 7, 2023
Maintainer

Hi @anujpb. The image build process is still a manual process that takes quite a while but it does appear that some images (elyra/enterprise-gateway, elyra/kernel-tf-py, and elyra/kernel-tf-gpu-py) did not complete the build and fell through the cracks. We apologize.

Since you've already pinged @lresende, I will ping him directly and see if he's able to complete the images for 3.2.2.

1 reply

lresende Apr 10, 2023
Maintainer

will re-run the build, sorry for the inconvenience...

lresende · 2023-04-10T17:59:25Z

lresende
Apr 10, 2023
Maintainer

Done, the kerne-tf-gpu-py for now is a problem to publish multi-arch, so I just publish x86.

0 replies

kevin-bates · 2023-04-10T18:07:07Z

kevin-bates
Apr 10, 2023
Maintainer

Thank you @lresende!

the kerne-tf-gpu-py for now is a problem to publish multi-arch, so I just publish x86.

Yes, that sounds familiar - no problem.

0 replies

kevin-bates · 2023-04-10T18:09:35Z

kevin-bates
Apr 10, 2023
Maintainer

Just noticed we haven't closed any discussions so, when this is closed, it kinda falls off the current set of discussions - which feels unhelpful to folks - so I'm going to reopen until we decide to start closing discussions that warrant their closure.

1 reply

anujpb Apr 10, 2023

Sounds good. I see that the x86 variants of the elyra/kernel-tf-py and elyra/kernel-tf-gpu-py images haven't made it to docker hub yet. In the future, we probably will require those too.

For now, I am surely happy about the elyra/enterprise-gateway image.
Many thanks to you @kevin-bates & @lresende.

paf91 · 2024-03-30T20:02:58Z

paf91
Mar 30, 2024

@kevin-bates @lresende does look that now 3.2.3 images are missing after a month of 3.2.3 release

0 replies

lresende · 2024-03-30T21:08:25Z

lresende
Mar 30, 2024
Maintainer

This was fixed, see #1371

1 reply

paf91 Mar 30, 2024

Unfortunately I'm not sure about that :(

lresende · 2024-03-30T21:17:02Z

lresende
Mar 30, 2024
Maintainer

I went to docker hub and it seems i built them all but never pushed, pushing them up now...

1 reply

paf91 Mar 30, 2024

thanks. could you please also tell me how I can run spark 3 using this version? seems to support 2.4 only...

lresende · 2024-03-30T22:13:59Z

lresende
Mar 30, 2024
Maintainer

You should be able to run Spark 3 with this version/release

0 replies

lresende · 2024-03-30T22:14:16Z

lresende
Mar 30, 2024
Maintainer

Feel free to close if this solve your issues

3 replies

paf91 Mar 30, 2024

im getting exec /usr/bin/tini: exec format error error using this image. what architecture it's using?

paf91 Mar 30, 2024

in the meantime 3.2.2 seem to start fine using (same command with 3.2.3 gives error exec /usr/bin/tini: exec format error)

helm  upgrade --install  enterprise-gateway \
  https://github.com/jupyter-server/enterprise_gateway/releases/download/v3.2.2/jupyter_enterprise_gateway_helm-3.2.2.tar.gz \
   --namespace enterprise-gateway \
   -f jupyter/gateway/values.yaml

values.yaml contents:

service:
  type: "ClusterIP"
  ports:
    # The primary port on which Enterprise Gateway is servicing requests.
    - name: "http"
      port: 8888
      targetPort: 8888
    # The  port on which Enterprise Gateway will receive kernel connection info responses.
    - name: "http-response"
      port: 8877
      targetPort: 8877
deployment:
  # Update CPU/Memory as needed
  resources:
    limits:
      cpu: 2
      memory: 10Gi
    requests:
      cpu: 1
      memory: 2Gi
  # Update to deploy multiple replicas of EG.
  replicas: 2
  # Give Enteprise Gateway some time to gracefully shutdown
  terminationGracePeriodSeconds: 60

kip:
  enabled: false # turn this off, if running DaemonSets is restricted by your cluster Administrator

ingress:
  enabled: true
  # Ingress resource host
  hostName: "[unique-fully-qualified-domain-name]"

paf91 Mar 30, 2024

@lresende please add linux/amd64 architecture for image release

lresende · 2024-03-31T19:17:29Z

lresende
Mar 31, 2024
Maintainer

@paf91 I left it building and pushed multi-arch images overnight, please try again and let me know if it works ok now...

3 replies

paf91 Apr 1, 2024

Cool, thanks!
Could you please tell me what could've I configured wrong:
When choosing Spark Python Kubernetes mode everything seems to work fine, but when I'm choosing Python Spark Operator i'm getting this error (i've configured spark operator to use webhooks):

HTTP 500: Internal Server Error (Error from Gateway: [KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file] KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file. Ensure gateway url is valid and the Gateway instance is running.)
[W 2024-04-01 11:25:54.537 ServerApp] wrote error: "HTTP 500: Internal Server Error (Error from Gateway: [KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file] KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file. Ensure gateway url is valid and the Gateway instance is running.)"
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/gateway/gateway_client.py", line 764, in gateway_request
        response = await rhc.fetch(endpoint, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/gateway/gateway_client.py", line 716, in fetch
        return await self._fetch(endpoint, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/gateway/gateway_client.py", line 728, in _fetch
        raise e
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/gateway/gateway_client.py", line 724, in _fetch
        response: HTTPResponse = await self.client.fetch(endpoint, **kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    tornado.httpclient.HTTPClientError: HTTP 500: KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/sessions/handlers.py", line 89, in post
        s_model = await sm.create_session(
                  ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/sessions/sessionmanager.py", line 285, in create_session
        kernel_id = await self.start_kernel_for_session(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/sessions/sessionmanager.py", line 343, in start_kernel_for_session
        kernel_id = await self.kernel_manager.start_kernel(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/gateway/managers.py", line 84, in start_kernel
        await km.start_kernel(kernel_id=kernel_id, **kwargs)
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/kernels/kernelmanager.py", line 778, in wrapped_method
        raise err
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/kernels/kernelmanager.py", line 742, in wrapped_method
        out = await method(self, *args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/gateway/managers.py", line 508, in start_kernel
        response = await gateway_request(
                   ^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/gateway/gateway_client.py", line 788, in gateway_request
        raise web.HTTPError(
    tornado.web.HTTPError: HTTP 500: Internal Server Error (Error from Gateway: [KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file] KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file. Ensure gateway url is valid and the Gateway instance is running.)

The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
                 ^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/auth/decorator.py", line 73, in inner
        return await out
               ^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/jupyter_server/services/sessions/handlers.py", line 107, in post
        raise web.HTTPError(500, str(e)) from e
    tornado.web.HTTPError: HTTP 500: Internal Server Error (HTTP 500: Internal Server Error (Error from Gateway: [KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file] KernelID: '87054a7a-5204-451a-b127-ad9bdb4b4923' launch timeout due to: Waited too long (40.0s) to get connection file. Ensure gateway url is valid and the Gateway instance is running.))
[E 2024-04-01 11:25:54.542 ServerApp] {
      "X-Forwarded-Host": "<redacted>",
      "X-Forwarded-Proto": "https",
      "X-Forwarded-Port": "443",
      "X-Forwarded-For": "::ffff:<redacted>",
      "Cookie": "<redacted>",
      "Accept-Language": "en-US,en;q=0.9",
      "Accept-Encoding": "gzip, deflate, br, zstd",
      "Referer": "<redacted>/lab/tree/<redacted>.ipynb",
      "Sec-Fetch-Dest": "empty",
      "Sec-Fetch-Mode": "cors",
      "Sec-Fetch-Site": "same-origin",
      "Origin": "<redacted>",
      "Accept": "*/*",
      "Sec-Ch-Ua-Platform": "\"Windows\"",
      "User-Agent": "<redacted>",
      "Authorization": "token [secret]",
      "Sec-Ch-Ua-Mobile": "?0",
      "X-Xsrftoken": "<redacted>",
      "Content-Type": "text/plain;charset=UTF-8",
      "Sec-Ch-Ua": "\"Google Chrome\";v=\"123\", \"Not:A-Brand\";v=\"8\", \"Chromium\";v=\"123\"",
      "Content-Length": "132",
      "Connection": "keep-alive",
      "Host": "<redacted>"
    }
[E 2024-04-01 11:25:54.542 ServerApp] 500 POST <redacted>/api/sessions?1711970713703 (<redacted>@::ffff:<redacted>) 41047.96ms

lresende Apr 1, 2024
Maintainer

This is not an easy one, do you have the spark operator, do you have the correct crd template/scripts on kernelspecs, etc.etc? You could check the crd submitted and it should have a status and reason for that status.... but this only tells you the kernel did not get started in the timeout period

paf91 Apr 4, 2024

yeah you're right but the thing, i see sparkapplication being created, but no CRDs and nothing happens. I have spark operator, and use pretty standard Spark Operator kernel that comes with EG

lresende · 2024-04-01T22:09:09Z

lresende
Apr 1, 2024
Maintainer

BTW, could you open another thread for the crd different topic?

1 reply

paf91 Apr 3, 2024

I created this issue though it's about whole EG 3.2.3 release #1379
Kubernetes kernels just dont start

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing docker image for elyra/enterprise-gateway:3.2.2 on Dockerhub #1287

{{title}}

Replies: 12 comments 11 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Missing docker image for elyra/enterprise-gateway:3.2.2 on Dockerhub #1287

Replies: 12 comments · 11 replies

kevin-bates Apr 7, 2023 Maintainer

lresende Apr 10, 2023 Maintainer

lresende Apr 10, 2023 Maintainer

kevin-bates Apr 10, 2023 Maintainer

kevin-bates Apr 10, 2023 Maintainer

lresende Mar 30, 2024 Maintainer

lresende Mar 30, 2024 Maintainer

lresende Mar 30, 2024 Maintainer

lresende Mar 30, 2024 Maintainer

lresende Mar 31, 2024 Maintainer

lresende Apr 1, 2024 Maintainer

lresende Apr 1, 2024 Maintainer

Replies: 12 comments 11 replies

kevin-bates
Apr 7, 2023
Maintainer

lresende Apr 10, 2023
Maintainer

lresende
Apr 10, 2023
Maintainer

kevin-bates
Apr 10, 2023
Maintainer

kevin-bates
Apr 10, 2023
Maintainer

lresende
Mar 30, 2024
Maintainer

lresende
Mar 30, 2024
Maintainer

lresende
Mar 30, 2024
Maintainer

lresende
Mar 30, 2024
Maintainer

lresende
Mar 31, 2024
Maintainer

lresende Apr 1, 2024
Maintainer

lresende
Apr 1, 2024
Maintainer