State Management and Caching

[shreyasbhat0]

While executing packages if there is a point of failure ,we need to run the whole package again . Instead if we can caching it will be easier to execute the package again from where the execution failed and even we wont miss the data from previous execution by using state where package can store necessary data which might be used in further execution as a key value pair.

Whats your thoughts on having something like Context based execution ? something similar to this

[mieubrisse]

Thanks for reporting this @shreyasbhat0 ! This is a pain point we've seen a couple times now, and @gbouv is actually working on something right now to let Kurtosis skip steps that it's already done: #741 Stay tuned for more soon!

[leeederek]

The work was moved to #769, which has now merged. As of Kurtosis 0.80.3, idempotent runs are now possible! Meaning subsequent runs of the same Starlark package will only perform the steps that have changed since the last run (while skipping the unchanged parts of the Plan).

So for your use case, let's say you ran a Starlark package with 4 steps, and it failed on step 3. You can fix things in step 3, re-run the Starlark package, and Kurtosis will skip steps 1 and 2 and perform only steps 3 and 4 (within the same enclave).

Going to mark this question as answered and closed!