Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Artf/triggers #4394

Merged
merged 4 commits into from
Nov 10, 2023
Merged

Artf/triggers #4394

merged 4 commits into from
Nov 10, 2023

Conversation

wild-endeavor
Copy link
Contributor

@wild-endeavor wild-endeavor commented Nov 9, 2023

pr into 4077. This PR adds the trigger concept.

Other changes:

  • Forgot to make the index on artifactkey unique.
  • Fix to create artifact, it wasn't pulling back out the artifact key.
  • Updated the Artifact service to handle the passing of artifacts created by the events processor to the trigger handling components. Currently, and this is potentially a design flaw, the events processor has no way of contacting the trigger component directly.

Signed-off-by: Yee Hing Tong <[email protected]>
…e check public, query for associated models before using because gorm doesn't work

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Copy link

codecov bot commented Nov 9, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (artifacts@6792f4e). Click here to learn what that means.

❗ Current head d1ced5c differs from pull request most recent head 4bab22f. Consider uploading reports for the commit 4bab22f to get more accurate results

Additional details and impacted files
@@             Coverage Diff              @@
##             artifacts    #4394   +/-   ##
============================================
  Coverage             ?   59.13%           
============================================
  Files                ?      369           
  Lines                ?    25595           
  Branches             ?        0           
============================================
  Hits                 ?    15135           
  Misses               ?     9019           
  Partials             ?     1441           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Yee Hing Tong <[email protected]>
@wild-endeavor wild-endeavor merged commit ee03489 into artifacts Nov 10, 2023
26 of 33 checks passed
@wild-endeavor wild-endeavor deleted the artf/triggers branch November 10, 2023 08:41
eapolinario added a commit that referenced this pull request Dec 27, 2023
* use-personal

* tidy

* update port to 30004

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bundle (#7)

* Remove private repos (#4078)

Signed-off-by: Yee Hing Tong <[email protected]>

* nit from monorepo merge

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bring in idl (#4204)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bring admin monorepo (#4203)

Main pr to bring in the boilerplate code for artifact service into the main pr.

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Artf/triggers (#4394)

This PR adds the trigger concept.

Other changes:
* Forgot to make the index on artifactkey unique.
* Fix to create artifact, it wasn't pulling back out the artifact key.
* Updated the Artifact service to handle the passing of artifacts created by the events processor to the trigger handling components. Currently, and this is potentially a design flaw, the events processor has no way of contacting the trigger component directly.

Signed-off-by: Yee Hing Tong <[email protected]>

* make trigger key index unique (#4410)

Signed-off-by: Yee Hing Tong <[email protected]>

* revert data proxy changes (#4411)

Signed-off-by: Yee Hing Tong <[email protected]>

* artf/ fix issue with trigger (#4412)

Signed-off-by: Yee Hing Tong <[email protected]>

* sandbox lite dockerfile (#4413)

Signed-off-by: Yee Hing Tong <[email protected]>

* More pgconn handling fixes (#4420)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/switch event (#4428)

Most of the changes here are generated proto changes. Actual code changes are:
IDL
* Remove the supplemental fields in the CloudEventTaskExecution object and move them to CloudEventNodeExecution object.
* Remove some fields that the artifact service ended up not using (parent_node_execution and scheduled_at)

in the cloudevent publisher, change the code filling in of the aforementioned supplemental information to happen for node execution events instead of task execution events.
* Remove the deleted fields.

On the event handling side, move the logic to the handling of the node event instead of the task event.
Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/lints (#4429)

* lint fixes
* rename sandbox_utils without underscore

Signed-off-by: Yee Hing Tong <[email protected]>

* look for other places where pgconn is cast and add handling (#4430)

Add additional handling of pgconn
* Change datacatalog logic to use flytestdlib function instead.
* Change flytestdlib constants to capitalize
* Add handling of the other type to Admin code directly

Signed-off-by: Yee Hing Tong <[email protected]>

* add artifacts branch to buf push (#4448)

Signed-off-by: squiishyy <[email protected]>

* add dummy comment (#4451)

Signed-off-by: squiishyy <[email protected]>

* add dummy comment again (#4453)

Signed-off-by: squiishyy <[email protected]>

* artf/updates to source (#4443)


Signed-off-by: Yee Hing Tong <[email protected]>

* add a feature gate (#4472)


Signed-off-by: Yee Hing Tong <[email protected]>

* fix bug in execution_manager with lpExpectedInputs, remove the standalone artifact client and move into idl clientset, update local config file, remove a deprecated function (#4473)

Signed-off-by: Yee Hing Tong <[email protected]>

* initial deletion of artifact code

Signed-off-by: Yee Hing Tong <[email protected]>

* running make manifests in docker/sandbox-bundled to update generated files

Signed-off-by: Yee Hing Tong <[email protected]>

* now compiles (#4481)

Signed-off-by: Yee Hing Tong <[email protected]>

* add a debug line

Signed-off-by: Yee Hing Tong <[email protected]>

* more debugging

Signed-off-by: Yee Hing Tong <[email protected]>

* clean up go mod and switch the order of the input extraction to check for uri first

Signed-off-by: Yee Hing Tong <[email protected]>

* change artifacts http mappings in protos

Signed-off-by: squiishyy <[email protected]>

* testing removal of go mod pin

Signed-off-by: Yee Hing Tong <[email protected]>

* update go mod some more

Signed-off-by: Yee Hing Tong <[email protected]>

* copilot go mod

Signed-off-by: Yee Hing Tong <[email protected]>

* change artifacts http mappings again (#4562)

Signed-off-by: squiishyy <[email protected]>

* panic and revert gorm log level change

Signed-off-by: Yee Hing Tong <[email protected]>

* revert local to 5

Signed-off-by: Yee Hing Tong <[email protected]>

* make helm

Signed-off-by: Yee Hing Tong <[email protected]>

* Reclaim space prior to running functional tests in single-binary

Signed-off-by: Eduardo Apolinario <[email protected]>

* Pin flytectl version to 0.8.0

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Pin flytectl version to 0.8.0"

This reverts commit cfa4745.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use flytagent v1.10.2

Signed-off-by: Eduardo Apolinario <[email protected]>

* Debug single-binary

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use an infinite loop instead of watch (term env var is not set)

Signed-off-by: Eduardo Apolinario <[email protected]>

* Get logs from single-binary pod

Signed-off-by: Eduardo Apolinario <[email protected]>

* Debug single-binary pod: use a different way of choosing the pod

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix flytectl version

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use port 5432 to connect to postgres

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Use port 5432 to connect to postgres"

This reverts commit 8ded67b.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Fix flytectl version"

This reverts commit 3968d40.

Signed-off-by: Eduardo Apolinario <[email protected]>

* fix database error

Signed-off-by: Yee Hing Tong <[email protected]>

* revert .github/workflows/single-binary.yml by pulling from master

Signed-off-by: Yee Hing Tong <[email protected]>

* Review feedback

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: squiishyy <[email protected]>
Co-authored-by: Joe Eschen <[email protected]>
Co-authored-by: squiishyy <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant