Disable agent-service plugin in binary helm chart since service agent is disabled by default #5459
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sabled by default Signed-off-by: Jason Parraga <[email protected]>
Signed-off-by: Jason Parraga <[email protected]>
Sovietaced
force-pushed
the
fix-sandbox-experience
branch
from
7dd3ddb to
52f13a8
Compare
Sovietaced
commented
Jun 8, 2024
| @@ -138,7 +138,7 @@ go-tidy: | |||
|
|
|||
| .PHONY: lint-helm-charts | |||
| lint-helm-charts: | |||
| # This pressuposes that you have act installed | |||
| # This presupposes that you have act installed | |||
There was a problem hiding this comment.
Just a minor typo I noticed while trying to lint the helm stuff
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5459 +/- ##
==========================================
- Coverage 61.07% 61.07% -0.01%
==========================================
Files 793 793
Lines 51226 51226
==========================================
- Hits 31288 31287 -1
- Misses 17062 17063 +1
Partials 2876 2876
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
After this PR is merged, we can enable the |
Fair enough, I will back out the helm chart change. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
This fixes an issue found where the Flyte sandbox workflow causes propeller to panic on startup. The cause of this panic is that propeller is attempting to initialize the agent plugin when the agent service is down (by default). Requests to the agent service to retrieve the list of supported tasks fails with a timeout.
{"json":{"src":"entrypoint.go:56"},"level":"info","msg":"Started propeller webhook","ts":"2024-06-08T00:43:17Z"} panic: failed to initialize agent registry with error: failed to list agent: [&{dns:///flyteagent.flyte.svc.cluster.local:8000 true map[gettask:10s] 10s}] with error: [rpc error: code = DeadlineExceeded desc = context deadline exceeded] goroutine 32 [running]: github.com/flyteorg/flyte/flyteplugins/go/tasks/plugins/webapi/agent.newAgentPlugin() /flyteorg/build/flyteplugins/go/tasks/plugins/webapi/agent/plugin.go:378 +0x45a github.com/flyteorg/flyte/flyteplugins/go/tasks/plugins/webapi/agent.RegisterAgentPlugin() /flyteorg/build/flyteplugins/go/tasks/plugins/webapi/agent/plugin.go:403 +0xa5 github.com/flyteorg/flyte/flytepropeller/pkg/controller/nodes/task.WranglePluginsAndGenerateFinalList.func1() /flyteorg/build/flytepropeller/pkg/controller/nodes/task/plugin_config.go:28 +0xf sync.(*Once).doSlow(0x0?, 0x1000100?) /usr/local/go/src/sync/once.go:74 +0xbf sync.(*Once).Do(...) /usr/local/go/src/sync/once.go:65 github.com/flyteorg/flyte/flytepropeller/pkg/controller/nodes/task.WranglePluginsAndGenerateFinalList({0x4302f80, 0xc00089ad70}, 0xc000773980, {0x7fc2084a7118, 0x5f3af40}, {0x43359c8?, 0xc00137f520}) /flyteorg/build/flytepropeller/pkg/controller/nodes/task/plugin_config.go:28 +0xac github.com/flyteorg/flyte/flytepropeller/pkg/controller/nodes/task.(*Handler).Setup(0xc000903450, {0x4302f80, 0xc00089ad70}, {0x42f6930?, 0xc0009d2600}) /flyteorg/build/flytepropeller/pkg/controller/nodes/task/handler.go:231 +0x191 github.com/flyteorg/flyte/flytepropeller/pkg/controller/nodes/factory.(*handlerFactory).Setup(0xc000157b80, {0x4302f80, 0xc00089ad70}, {0x430f480?, 0xc000816320}, {0x42f6930, 0xc0009d2600}) /flyteorg/build/flytepropeller/pkg/controller/nodes/factory/handler_factory.go:73 +0x81f github.com/flyteorg/flyte/flytepropeller/pkg/controller/nodes.(*recursiveNodeExecutor).Initialize(0xc000816320, {0x4302f80, 0xc00089ad70}) /flyteorg/build/flytepropeller/pkg/controller/nodes/executor.go:464 +0xbf github.com/flyteorg/flyte/flytepropeller/pkg/controller/workflow.(*workflowExecutor).Initialize(0xc000482500, {0x4302f80, 0xc00089ad70}) /flyteorg/build/flytepropeller/pkg/controller/workflow/executor.go:385 +0x59 github.com/flyteorg/flyte/flytepropeller/pkg/controller.(*Propeller).Initialize(0x4302f80?, {0x4302f80?, 0xc00089ad70?}) /flyteorg/build/flytepropeller/pkg/controller/handler.go:83 +0x26 github.com/flyteorg/flyte/flytepropeller/pkg/controller.(*WorkerPool).Initialize(...) /flyteorg/build/flytepropeller/pkg/controller/workers.go:121 github.com/flyteorg/flyte/flytepropeller/pkg/controller.(*Controller).run(0xc0005871f0, {0x4302f80, 0xc00089ad70}) /flyteorg/build/flytepropeller/pkg/controller/controller.go:109 +0x82 github.com/flyteorg/flyte/flytepropeller/pkg/controller.(*Controller).Run(0xc0005871f0, {0x4302f80?, 0xc00089ad70}) /flyteorg/build/flytepropeller/pkg/controller/controller.go:96 +0x119 github.com/flyteorg/flyte/flytepropeller/pkg/controller.StartController({0x4302f80?, 0xc00089acd0?}, 0xc000830000, {0x343e1f4, 0x3}, {0x431e3c0?, 0xc00137e680}, 0xc001408000) /flyteorg/build/flytepropeller/pkg/controller/controller.go:612 +0x5b4 github.com/flyteorg/flyte/cmd/single.startPropeller.func2() /flyteorg/build/cmd/single/start.go:167 +0x94 golang.org/x/sync/errgroup.(*Group).Go.func1() /go/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:78 +0x56 created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 82 /go/pkg/mod/golang.org/x/sync@v0.6.0/errgroup/errgroup.go:75 +0x96What changes were proposed in this pull request?
This change remove the agent service from the enabled plugin list considering that the flyte agent service is disabled by default.
How was this patch tested?
See generated Kubernetes manifests.
Setup process
Screenshots
Check all the applicable boxes