Parallel Concurrent Build

[type-checker]

I am exploring nixpacks as an alternative of buildpacks. I have checked nixpacks doc, however didn't found answers for few queries and one of the most important question is:

How do we achieve parallel concurrent build on scale using nixpacks?

cc: @coffee-cup

[ndneighbor]

Going to tag in @nebulatgs who also might have some thoughts here.

[coffee-cup]

I'm slightly confused by what you mean @type-checker.

Nixpacks is a way to create an image from a code directory and does its best to infer language and system dependencies. Under the hood it is analyzing the code, creating a build plan that includes what to install (via Nix) and what commands to run for the build. It then creates and executes a Dockerfile, resulting in a image that can be distributed and run locally or on a number of cloud platforms.

What type of parallel builds are you doing? Are you trying to build multiple images at once?

[type-checker]

I'm slightly confused by what you mean @type-checker.

Nixpacks is a way to create an image from a code directory and does its best to infer language and system dependencies. Under the hood it is analyzing the code, creating a build plan that includes what to install (via Nix) and what commands to run for the build. It then creates and executes a Dockerfile, resulting in a image that can be distributed and run locally or on a number of cloud platforms.

What type of parallel builds are you doing? Are you trying to build multiple images at once?

Hi @coffee-cup, Thank you for answering!
Yes, when I am trying to build multiple images concurrently the build speed is becoming slow.
For ex. I tested with the same Next.js project:
single build took me 60sec.,
two concurrent build took 1min.20sec.
three concurrent build took 5 min.45sec.

So, may query is how should I scale the parallel concurrent build, so that I should achieve the same 60sec. no matter the no.s of builds increases.

[coffee-cup]

How are you running the concurrent builds?

[type-checker]

How are you running the concurrent builds?

So, I've tried in the same network of Docker and then different network for each build but both didn't worked. I've also tried with Kaniko then which creates separate pod for each build but again it was the same result, build started becoming more slow with the numbers of concurrency build.

Could you plz. suggest how to scale parallel concurrency build. I am new to the ecosystem.

[coffee-cup]

What you are trying to do is related to running multiple Docker builds at once at scale and is out of scope of Nixpacks and this discussion form. It also seems like something very application specific. I recommend looking for help in a dedicated programming help forum.