Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

display flyte deck in arraynode subnodes #6055

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
plumbing through deckURI
Signed-off-by: Daniel Rammer <[email protected]>
hamersaw committed Nov 28, 2024
commit 5964cc1f2fde30f4f91ed1120d88533cf36d6624
3 changes: 3 additions & 0 deletions flyteadmin/pkg/repositories/transformers/task_execution.go
Original file line number Diff line number Diff line change
@@ -309,6 +309,9 @@
existing.CacheStatus = latest.GetCacheStatus()
}
existing.Logs = mergeLogs(existing.GetLogs(), latest.GetLogs())
if latest.GetDeckUri() != "" && existing.GetDeckUri() != latest.GetDeckUri() {
existing.DeckUri = latest.GetDeckUri()
}

Check warning on line 314 in flyteadmin/pkg/repositories/transformers/task_execution.go

Codecov / codecov/patch

flyteadmin/pkg/repositories/transformers/task_execution.go#L313-L314

Added lines #L313 - L314 were not covered by tests

return existing
}
3 changes: 2 additions & 1 deletion flyteidl/gen/pb-es/flyteidl/event/event_pb.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion flyteidl/gen/pb-go/flyteidl/event/event.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion flyteidl/gen/pb_rust/flyteidl.event.rs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion flyteidl/protos/flyteidl/event/event.proto
Original file line number Diff line number Diff line change
@@ -288,7 +288,8 @@ message ExternalResourceInfo {
// log information for the external resource execution
repeated core.TaskLog logs = 6;

// TODO @hamersaw - docs
// String location uniquely identifying where the deck HTML file is
// NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)
string deck_uri = 7;
}

7 changes: 7 additions & 0 deletions flytepropeller/pkg/controller/nodes/array/event_recorder.go
Original file line number Diff line number Diff line change
@@ -92,13 +92,18 @@

// process events
cacheStatus := idlcore.CatalogCacheStatus_CACHE_DISABLED
var deckURI string
for _, nodeExecutionEvent := range e.nodeEvents {
switch target := nodeExecutionEvent.GetTargetMetadata().(type) {
case *event.NodeExecutionEvent_TaskNodeMetadata:
if target.TaskNodeMetadata != nil {
cacheStatus = target.TaskNodeMetadata.GetCacheStatus()
}
}

if len(nodeExecutionEvent.DeckUri) > 0 {
deckURI = nodeExecutionEvent.DeckUri
}

Check warning on line 106 in flytepropeller/pkg/controller/nodes/array/event_recorder.go

Codecov / codecov/patch

flytepropeller/pkg/controller/nodes/array/event_recorder.go#L105-L106

Added lines #L105 - L106 were not covered by tests
}

// fastcache will not emit task events for cache hits. we need to manually detect a
@@ -110,6 +115,7 @@
RetryAttempt: retryAttempt,
Phase: idlcore.TaskExecution_SUCCEEDED,
CacheStatus: cacheStatus,
DeckUri: deckURI,

Check warning on line 118 in flytepropeller/pkg/controller/nodes/array/event_recorder.go

Codecov / codecov/patch

flytepropeller/pkg/controller/nodes/array/event_recorder.go#L118

Added line #L118 was not covered by tests
})
}

@@ -143,6 +149,7 @@
RetryAttempt: retryAttempt,
Phase: taskExecutionEvent.GetPhase(),
CacheStatus: cacheStatus,
DeckUri: deckURI,
})
}

Loading