Deployer types #117

mfleader · 2023-10-27T14:48:38Z

Changes introduced with this PR

Adjust deployer based on parameters in workflow plugin.

By contributing to this repository, I agree to the contribution guidelines.

Also improved debug messages

Mostly this is test cases, and some mild refactoring that should have a similar result

This ensures that the goroutines are done when the engine exits. This fixes a test case, and will be important when retrying is added to the deadlock check

This is a workaround for the situation where all of the steps are processing at the same time, all waiting for input

Also improved error logic

Required for prior commits to work

This adds the done message to the SDK

This means new error handling, so multiple errors can be reported

Force stop is good for when you do not need to wait for the ATP client, or something equivalant, to finish. Waiting for the ATP to finish means waiting for the steps to finish entirely.

I reduced the amount of code in one function, and reduced redundancy of unlocking

These ones were only shown on CI, not locally

internal/step/dummy/provider_test.go

internal/step/foreach/provider.go

internal/step/lifecycle.go

internal/step/plugin/provider.go

internal/step/plugin/provider_test.go

workflow/executor.go

workflow/model.go

workflow/workflow.go

jaredoconnell

The name/ID refactor made changes out of scope.
In addition, we should probably add a test for invalid deployer ID. I'll do that.

internal/step/plugin/provider.go

internal/step/lifecycle.go

internal/step/foreach/provider.go

internal/step/dummy/provider_test.go

internal/step/plugin/provider.go

internal/yaml/parser.go

workflow/executor.go

workflow/model.go

…nt configs

The docker deployer is already used extensively in the tests. No need to have a second deployer that also needs to be configured

jaredoconnell

Looks good to me. Tested locally.

jaredoconnell and others added 30 commits August 4, 2023 10:16

Added workflow pre-validation

c6fc1cd

Also improved debug messages

Updated go mod/sum

5e5659a

Progress towards debugging deadlock detection

25fa40e

Mostly this is test cases, and some mild refactoring that should have a similar result

Use image that supports ARM64

8188317

Ensure that run finishes before closing the step

e715add

This ensures that the goroutines are done when the engine exits. This fixes a test case, and will be important when retrying is added to the deadlock check

Added deadlock check retries

ac0ef7e

This is a workaround for the situation where all of the steps are processing at the same time, all waiting for input

Update test cases to take less time

c5696ec

Also improved error logic

Update dependencies

4c72b39

Required for prior commits to work

Update go SDK to dev branch

54503ae

This adds the done message to the SDK

Updated test deployer to dev branch

9ad0570

Updated deployers, and added python deployer to run-plugin

1770f19

Label loggers

2fe30be

Update to use ATP v3

cb57e6c

This means new error handling, so multiple errors can be reported

Update SDK in go mod, and add force stop to step interface

30b9c23

Force stop is good for when you do not need to wait for the ATP client, or something equivalant, to finish. Waiting for the ATP to finish means waiting for the steps to finish entirely.

Merge branch 'main' into atp-v2

9088d54

Cleanup

4f87b19

Fix tests

e1768b5

Update SDK and test deployer

f165777

Reduce redundancy in Plugin Provider

d8c64bc

Refactoring to fix linting errors

ce8c752

I reduced the amount of code in one function, and reduced redundancy of unlocking

Fix linting errors

ec46f7b

These ones were only shown on CI, not locally

bump internal dependencies

58e5bf0

update all internal dependencies

3fe61d5

refactor config to use multiple local deployers

15c4fce

config defaults

dbc3fb8

fix broken tests

6f1cf65

fix broken tests

0e9e558

update plugin schema with src and type keys

5d9d4f6

fix merge conflicts

0ed414e

remove duplicated test

c09b7ba

jaredoconnell and others added 12 commits October 27, 2023 17:46

Progress towards implementation of deployment types

a8894dd

getting the static type of an object comes out in golang 1.22

a69ea86

type switch on unserialized workflow deployer config

aeb097b

remove dead code

880866f

use pointers

f979169

default to builtin deployer stub

c164285

updated plugin provider schema

d4f42ee

fix some broken tests

70942df

fix more broken tests

cc9f0d2

Call close to improve error reporting in Python deployer

de794a2

remove dead code

b794404

fix lint and style

488489d

mfleader marked this pull request as ready for review November 1, 2023 14:02

refactor to use deployer_name as discriminator value for factories

32bbcff

mfleader requested a review from jaredoconnell November 1, 2023 15:07

update deployer dependencies

f8482c2

mfleader self-assigned this Nov 1, 2023

dbutenhof reviewed Nov 1, 2023

View reviewed changes

Matt Leader added 4 commits November 1, 2023 17:16

fix error joining

a9e4368

unfactor Name back to ID

7cb26db

unfactor more Name to ID

30bb6f7

update to correct kubernetes deployer version

d60986f

jaredoconnell requested changes Nov 1, 2023

View reviewed changes

Matt Leader and others added 4 commits November 1, 2023 18:15

fix default value

8901b4b

Added tests to validate that the workflow fails with invalid deployme…

e42addf

…nt configs

Bump workflow version, and switch from Python to Docker deployer

ce9c4bb

The docker deployer is already used extensively in the tests. No need to have a second deployer that also needs to be configured

update readme

6cfcfa8

jaredoconnell approved these changes Nov 1, 2023

View reviewed changes

mfleader merged commit 9054fdf into main Nov 1, 2023
5 checks passed

mfleader deleted the deployer-types branch November 1, 2023 22:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deployer types #117

Deployer types #117

mfleader commented Oct 27, 2023

jaredoconnell left a comment

jaredoconnell left a comment

Deployer types #117

Deployer types #117

Conversation

mfleader commented Oct 27, 2023

Changes introduced with this PR

jaredoconnell left a comment

Choose a reason for hiding this comment

jaredoconnell left a comment

Choose a reason for hiding this comment