Clean up agents from pool when Terraform destroy is called

[jcorioland]

Currently, if you call the terraform destroy commands, all the containers instances are removed, but the agents are still present in the Azure DevOps agents pool. They are marked as offline, so no job can be scheduled on it, but still there and have to be removed manually.

We should find a way to clean up automatically when terraform destroy is called. Maybe using a local executer or something like that.

[mozts2005]

@jcorioland I have fixed this in use case and would be happy to add a PR for the fix.

[jcorioland]

@mozts2005 awesome, I am happy to get contributions on the project! How did you fix this in your use case? Looking forward for the PR. Thanks!

[mozts2005]

I did this by adding the Azure DevOps provider and creating the Pool in the state file.

[mozts2005]

I have been able to remove individual agents in AKS clusters when the containers get a safe shutdown by updating the startup.sh script to call the cleanup function. but I have not been able to get an ACI to perform a safe shutdown.

[jcorioland]

@mozts2005
I did this by adding the Azure DevOps provider and creating the Pool in the state file.
On this one, I think you are talking about #4 and not this issue. I have a work in progress on this one, and indeed I am using the new Azure DevOps provider.

I have been able to remove individual agents in AKS clusters when the containers get a safe shutdown by updating the startup.sh script to call the cleanup function. but I have not been able to get an ACI to perform a safe shutdown.

This is exactly my problem, I cannot find a way to gracefully shutdown an ACI container.

[yuri-tieto]

I have been able to remove individual agents in AKS clusters when the containers get a safe shutdown by updating the startup.sh script to call the cleanup function. but I have not been able to get an ACI to perform a safe shutdown.

@mozts2005 can you share the code for calling cleanup of the agent? I see reference to config.sh, but can't find it inside this repository.

[mozts2005]

add this to the bottom of the start.sh

print_header "2. Running Azure Pipelines agent..."

# `exec` the node runtime so it's aware of TERM and INT signals
# AgentService.js understands how to handle agent self-update and restart
exec ./externals/node/bin/node ./bin/AgentService.js interactive --once & wait $!

# We expect the above process to exit when it runs once,
# so we now run a cleanup process to remove this agent
# from the pool
cleanup

the cleanup function can be viewed here https://github.com/Azure/terraform-azurerm-aci-devops-agent/blob/master/docker/linux/start.sh#L31

the config.sh is part of the agent that is installed by the startup.sh

[mozts2005]

Are there any changes need to PR #11?

[joshjohanning]

@jcorioland did the above PR solve the cleanup issue on destroy (not sure if you tested it yourself); is this something you were still pursuing?

[jcorioland]

@soccerjoshj07 for some reason it has not been finished. I am looking at it right now to finish end to end tests and merge. Thanks!

[bitsofinfo]

any update on this?