Add support for dynamic pipeline runs with persistence #12367
Conversation
|
I see that you haven't updated any README files. Would it make sense to do so? |
vyzaldysanchez
requested review from
jmank88,
cedric-cordenier and
george-dorin
| FROM public.jobs; | ||
|
|
||
|
|
||
| ALTER TABLE public.jobs DROP COLUMN pipeline_spec_id; -- Do we use CASCADE here? Does it have any relationship with other tables? |
There was a problem hiding this comment.
I would hold off on dropping this column until a later version -- might be better to deprecate the field first. I might be overly cautious here (I know that we recommend NOPs take a DB backup, but I don't think they all do this reliably.)
@jmank88 Is this a worthwhile precaution to take or not worth it?
There was a problem hiding this comment.
bumping @jmank88
There was a problem hiding this comment.
Hmm I don't like the idea of compromising to make up for a lack of backups 🤔
It would be neat if we could do that reliably and generally, but I think there would be many difficulty, messy, and impossible cases. Is this case uniquely important?
There was a problem hiding this comment.
I don't think there's anything uniquely important for this field in specific really.
| @@ -48,7 +49,8 @@ type Runner interface { | |||
| // ExecuteAndInsertFinishedRun executes a new run in-memory according to a spec, persists and saves the results. | |||
| // It is a combination of ExecuteRun and InsertFinishedRun. | |||
| // Note that the spec MUST have a DOT graph for this to work. | |||
| ExecuteAndInsertFinishedRun(ctx context.Context, spec Spec, vars Vars, l logger.Logger, saveSuccessfulTaskRuns bool) (runID int64, finalResult FinalResult, err error) | |||
| // This will persist the Spec in the DB if it doesn't have an ID. | |||
vyzaldysanchez
requested review from
cedric-cordenier
and removed request for
a team
|
* develop: Update Nix apple sdk usage (#12758) [KS-136] Update staging (#12703) update changelog path (#12757) Add support for dynamic pipeline runs with persistence (#12367) Fix nix mac issues building core missing newer apple sdk libs (#12746) Fix issues in package.json (#12748) [fix] Install jq when building plugins image (#12750) Automatic gas in load tests (#12416) remove VERSION file to use version from package.json (#12663) update package.json version (#12745)
This PR looks to provide support for dynamic pipeline runs with persistence, by changing the DB structure and adding a new relationship table:
job_pipeline_spec.job_pipeline_spec.