[cloud] - issues connecting via cli

[barnabasbusa]

What's your CLI version?

0.85.53

Description & steps to reproduce

INFO[2023-12-21T16:34:17+01:00] Loading cloud instance 02a311f602d04a72a97d007cac0a0c5b 
INFO[2023-12-21T16:34:17+01:00] Successfully Loaded API Key...               
INFO[2023-12-21T16:34:18+01:00] Adding new context 'cloud-02a311f602d04a72a97d007cac0a0c5b' 
INFO[2023-12-21T16:34:18+01:00] Context successfully added                   
INFO[2023-12-21T16:34:18+01:00] Switching context...                         
INFO[2023-12-21T16:34:19+01:00] Upgrading currently Kurtosis Portal version '0.0.4' to '0.0.7' 
INFO[2023-12-21T16:34:26+01:00] Kurtosis Portal started successfully on PID 18441 
INFO[2023-12-21T16:34:26+01:00] Context set to 'cloud-02a311f602d04a72a97d007cac0a0c5b', Kurtosis engine will now be restarted 
INFO[2023-12-21T16:34:26+01:00] No Kurtosis engine was found; attempting to start one... 
WARN[2023-12-21T16:34:35+01:00] The context was successfully set to 'cloud-02a311f602d04a72a97d007cac0a0c5b' but Kurtosis failed to start an engine in this new context. A new engine should be started manually with 'kurtosis engine restart'. The error was:
An error occurred starting the engine with the engine existence guarantor
 --- at /root/project/cli/cli/helpers/engine_manager/engine_manager.go:202 (EngineManager.StartEngineIdempotentlyWithDefaultVersion) ---
Caused by: An error occurred guaranteeing that a Kurtosis engine is running
 --- at /root/project/cli/cli/helpers/engine_manager/engine_manager.go:362 (EngineManager.startEngineWithGuarantor) ---
Caused by: An error occurred launching the engine server container
 --- at /root/project/cli/cli/helpers/engine_manager/engine_existence_guarantor.go:196 (engineExistenceGuarantor.VisitStopped) ---
Caused by: An error occurred launching the engine server container with default version tag '0.85.53'
 --- at /root/project/engine/launcher/engine_server_launcher/engine_server_launcher.go:70 (EngineServerLauncher.LaunchWithDefaultVersion) ---
Caused by: An error occurred launching the engine server container with environment variables 'map[SERIALIZED_ARGS:{"grpcListenPortNum":9710,"logLevelStr":"info","imageVersionTag":"0.85.53","metricsUserId":"2d01bb50fd304dea5d8356b294190f28ead3452cf91cfd3c7a80b2823e00dce7","didUserAcceptSendingMetrics":false,"kurtosisBackendType":"kubernetes","kurtosisBackendConfig":{"StorageClass":"hostpath"},"onBastionHost":true,"poolSize":0,"enclaveEnvVars":"{\"AWS_ACCESS_KEY_ID\": \"AKIATYNZLDGVKKJ6INJ3\", \"AWS_SECRET_ACCESS_KEY\": \"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\", \"AWS_BUCKET_REGION\": \"us-east-1\", \"AWS_BUCKET_NAME\": \"kurtosis-cloud\", \"AWS_BUCKET_USER_FOLDER\": \"cloud-user-7a5a4c805eeb4dc0887a1192418babdd\"}","is_ci":false,"cloud_user_id":"7a5a4c805eeb4dc0887a1192418babdd","cloud_instance_id":"02a311f602d04a72a97d007cac0a0c5b"}]'
 --- at /root/project/engine/launcher/engine_server_launcher/engine_server_launcher.go:129 (EngineServerLauncher.LaunchWithCustomVersion) ---
Caused by: An error occurred creating the engine using image 'kurtosistech/engine' with tag '0.85.53'
 --- at /root/project/container-engine-lib/lib/backend_impls/metrics_reporting/metrics_reporting_kurtosis_backend.go:63 (MetricsReportingKurtosisBackend.CreateEngine) ---
Caused by: An error occurred creating engine using image 'kurtosistech/engine:0.85.53', grpc port number '9710' and environment variables 'map[SERIALIZED_ARGS:{"grpcListenPortNum":9710,"logLevelStr":"info","imageVersionTag":"0.85.53","metricsUserId":"2d01bb50fd304dea5d8356b294190f28ead3452cf91cfd3c7a80b2823e00dce7","didUserAcceptSendingMetrics":false,"kurtosisBackendType":"kubernetes","kurtosisBackendConfig":{"StorageClass":"hostpath"},"onBastionHost":true,"poolSize":0,"enclaveEnvVars":"{\"AWS_ACCESS_KEY_ID\": \"AKIATYNZLDGVKKJ6INJ3\", \"AWS_SECRET_ACCESS_KEY\": \"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\", \"AWS_BUCKET_REGION\": \"us-east-1\", \"AWS_BUCKET_NAME\": \"kurtosis-cloud\", \"AWS_BUCKET_USER_FOLDER\": \"cloud-user-7a5a4c805eeb4dc0887a1192418babdd\"}","is_ci":false,"cloud_user_id":"7a5a4c805eeb4dc0887a1192418babdd","cloud_instance_id":"02a311f602d04a72a97d007cac0a0c5b"}]'
 --- at /root/project/container-engine-lib/lib/backend_impls/kubernetes/kubernetes_kurtosis_backend/kubernetes_kurtosis_backend.go:147 (KubernetesKurtosisBackend.CreateEngine) ---
Caused by: An error occurred creating the engine pod
 --- at /root/project/container-engine-lib/lib/backend_impls/kubernetes/kubernetes_kurtosis_backend/engine_functions/create_engine.go:133 (CreateEngine) ---
Caused by: An error occurred while creating the pod with name 'kurtosis-engine-122f347a140242b7bd71da90d15867e2' in namespace 'kurtosis-engine-122f347a140242b7bd71da90d15867e2' with image 'kurtosistech/engine:0.85.53'
 --- at /root/project/container-engine-lib/lib/backend_impls/kubernetes/kubernetes_kurtosis_backend/engine_functions/create_engine.go:441 (createEnginePod) ---
Caused by: An error occurred waiting for pod 'kurtosis-engine-122f347a140242b7bd71da90d15867e2' to become available
 --- at /root/project/container-engine-lib/lib/backend_impls/kubernetes/kubernetes_manager/kubernetes_manager.go:1157 (KubernetesManager.CreatePod) ---
Caused by: Pod 'kurtosis-engine-122f347a140242b7bd71da90d15867e2' failed before availability with the following state:
>>>>>>>>>>>>>>>>>>>>>>>>>> Pod kurtosis-engine-122f347a140242b7bd71da90d15867e2 <<<<<<<<<<<<<<<<<<<<<<<<<<
Container Statuses: - kurtosis-engine-container (docker.io/kurtosistech/engine:0.85.53): TERMINATED with exit code 1 - 

-------------------- Container kurtosis-engine-container Logs --------------------
2023-12-21T16:34:33.347426913+01:00 ERRO[2023-12-21T15:34:33Z][main.go:main] An error occurred when running the main function 
2023-12-21T16:34:33.347455479+01:00 The remote backend config '/run/engine/remote_backend_config.json' cannot be found
2023-12-21T16:34:33.347461625+01:00  --- at /home/circleci/project/engine/server/engine/main.go:161 (runMain) ---
2023-12-21T16:34:33.347466863+01:00 Caused by: open /run/engine/remote_backend_config.json: no such file or directory

------------------ End Container kurtosis-engine-container Logs ---------------------
>>>>>>>>>>>>>>>>>>>>>>>> End Pod kurtosis-engine-122f347a140242b7bd71da90d15867e2 <<<<<<<<<<<<<<<<<<<<<<<<<<<
 --- at /root/project/container-engine-lib/lib/backend_impls/kubernetes/kubernetes_manager/kubernetes_manager.go:1861 (KubernetesManager.waitForPodAvailability) --- 

Desired behavior

Be able to connect.

What is the severity of this bug?

Critical; I am blocked and Kurtosis is unusable for me because of this bug.

What area of the product does this pertain to?

Frontend: the Graphical User Interface, including the Enclave Manager

[adschwartz]

@barnabasbusa I think you may have switched your kurtosis cluster and need to switch it back. To confirm this is the case first run this command:

kurtosis cluster get

if it says docker then my hypothesis is wrong and this is not the issue. If it does not, but rather something regarding kubernetes, then switch over to k8s:

kurtosis cluster set docker

Background:
When connecting to you cloud instance, the kurtosis CLI needs to be configured to connect to a docker based backend (for now).

[barnabasbusa]

this has been since fixed. I was able to connect to the cluster, and update the engine.

I should be able to bump the engine without needing to connect to it via cli though.