Re-running a computational pipeline while a dynamic service is started can fail the dynamic service starting procedure

[sanderegg]

Is there an existing issue for this?

• I have searched the existing issues

Which deploy/s?

production aws (e.g. osparc.io), staging aws (e.g. staging.osparc.io), production on-premise (dalco), stating on-premise (dalco), development (master)

@AntoninoMarioC encountered this issue on 5.12.2024

Current Behavior

• Create a sleeper
• Connect a jupyter-math, ensure connection of the outputs of the sleeper to the jupyter-math is correctly setup
• run the sleeper
• once it ran, change the sleep interval to 120s
• re-run the computation
• start the jupyter-math

--> the jupyter-math fails to start

Expected Behavior

the jupyter-math should start

Steps To Reproduce

No response

Anything else?

The problem arises due to the following chain of event:

• the first run of the sleeper created some output files
• when the user re-run the sleeper with the longer interval:
  • dv-2 asks for the new set of upload links to storage,
  • storage starts by deleting the current file,
  • then returns the links
  • dv-2 starts the sleeper
• during that time the jupyter-math asks whether there are inputs, and it thinks there should be some, then tries to download them and fails, thus stops the jupyter-math

Current workaround

1. first start the dynamic service, then re-run the pipeline
2. or, start the pipeline and wait for the results to be uploaded, then start the jupyter-math

[sanderegg]

Possible solutions

1. storage deletes the file, which might not be necessary anymore since we now do not rely on it for unsuccessful multipart uploads (to be assessed)
2. dv-2 could explicitely require deletion of the outputs