How to optimize time spent on container creation?

[spesotskiy]

Hi All!

We have just started using AL-Go and have just a few tests to start with, so run pipeline actions usually take:

• Pulling generic image (6 min), Installation (4 min), Initialization, Creating container (5 min) ~ 15 min
• Importing Test Toolkit, Compiling apps (6 min), test apps and importing test toolkit, Publishing apps (2 min), Running tests (<1 min) ~ 9 min

I am wondering, how could I minimize the first part and avoid downloading a new image and creating a new container every time?
I think it would be enough to get a new container per a critical update (or even less frequent). So, I prepare the container manually once and then the pipeline picks it up until I update it again. Sounds like a plan!

What is the best place to store the prepared container? I am not sure that downloading that container will be faster than creating a new one...
And is it possible to skip "Pulling generic image" step or replace it with something faster?

I assume I need to override two scripts at least:

• DockerPull.ps1
• NewBcContainer.ps1

Any suggestions or samples of similar changes?

[jonaswre]

Hi, so the most straight forward solution to your problem is use a self hosted runner.

The self-hosted runner will keep images on disk so they are not re downloaded.

Self hosted runners can be hosted anywhere you can buy a VM in the cloud or use hardware on prem depending on your needs.

This reduced our pipeline runtimes to about 5 min total.

I had some discussion with @freddydk on this topic. But there are no real good solution. GitHub hosted runners will throw away everything downloaded and redownload it when they need it. Caching the right things is not trivial.

Freddy has created a script to setup a self hosted runner. That makes it straight forward to setup. We pay around 200$ per Month for a hosted Azure VM. But you could run that VM anywhere and probably cheaper.

[spesotskiy]

You mentioned "Freddy has created a script".
Is it this one https://github.com/microsoft/AL-Go/blob/main/Scenarios/SelfHostedGitHubRunner.md ?

So, let's say we have a VM and installed all tools listed by Freddy.
In order to skip "Pulling generic image" we need to install docker as well. Then I just add an empty DockerPull.ps1 to skip it, right? Or it will require more coding?
If the container is also prepared as a "template".In case of multiple jobs we need to create a new container per job, right?
Then in NewBcContainer.ps1 I would need to create a new container - a copy of the "template" one.

[jonaswre]

Yes that's the one.

There is no coding required.

Docker pull will first try to locate the image locally so in the first run it won't find it and download it. After that as long as the image stays the same it will use the local image. If a new minor is released, the first container after the minor was released will download it and every other container will then just use the already downloaded image.

There is an option to locally save a container after it was initialized. So saving it after the SQL Server and Service Tier was initialized. But I'm currently not sure if that's automatically configured or it doesn't make sense. Let's just say Freddy optimized a lot and for most use cases that's enough. Just install your own agent and enjoy.