parallelize insights gathering step and run it before the build completion

[shreddedbacon]

If its possible to run insights gathering in parallel, this would be really good to do. This step can take some time to run, so any reduction in time is welcome.

It should also probably be moved from being the last step of a build, to the second last step. Build completion should be the final step.

[tobybellwood]

Given that the insights run is just a docker container being used to scan another docker container, it should be ok to lightly parallel - but may have more implications if you have large numbers at once across multiple builds?

The reason it was placed after the build completion was to be able to ensure that the build completed (and scanning didn't delay anything else) - although with the current logic, the build still doesn't report until the scan has finished - maybe we introduce a post-build step instead and place the scan in there if it's configured?

[shreddedbacon]

Until insights is a separate process not tied to builds, it should be before the build completion.
Build completion means build completion, not build is completed but there is still more to do that isn't tied to the build but sort of is

[shreddedbacon]

I'm hesitant to add post-build unless there is a case for more things to be added to it, which is what post-deploy is really meant to be for, since it runs before the build completion.

If the build process is this, it is kind of silly

• build..
• post-deploy tasks
• completion
• post-completion tasks

[tobybellwood]

Yeah, good point. I'm conscious that this step (or indeed others) could add XX minutes to a build that for all intents and purposes is otherwise completed and is ready to use. If a situation arose that a build needed reverting/rerunning, it would be a pain to wait for the insights to complete. Maybe the SBOM should be a special task triggered by the build - it does need to be closely coupled to the build, to ensure that the images for that build are still in the docker_host.

EDIT: to add "special" task, as the SBOM task will need access to the docker host it may require further configuration. Having it run in the build ensures it has access.

[shreddedbacon]

But the build isn't complete until all tasks are complete :)

The build and deploy of the environment may technically be complete, but the build process itself is not.