Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: chore: Upgrade Argo to v3.4.16 #1

Open
wants to merge 113 commits into
base: kf-master-ref
Choose a base branch
from
Open

Conversation

gmfrasca
Copy link
Owner

Description of your changes:
Upgrade backend to Argo v3.4.16

Checklist:

droctothorpe and others added 30 commits February 16, 2024 19:42
PiperOrigin-RevId: 607802006
PiperOrigin-RevId: 609081795
PiperOrigin-RevId: 609107204
PiperOrigin-RevId: 609459972
… tolerations (kubeflow#10471)

* feat(Backend + SDK): Update kfp backend and kubernetes sdk to support tolerations

Signed-off-by: droctothorpe <[email protected]>
Co-authored-by: edmondop <[email protected]>
Co-authored-by: tarat44 <[email protected]>

* Address PR review 1

Signed-off-by: droctothorpe <[email protected]>
Co-authored-by: edmondop <[email protected]>
Co-authored-by: tarat44 <[email protected]>

* Refactor add_toleration to use Python primitives

Signed-off-by: droctothorpe <[email protected]>
Co-authored-by: edmondop <[email protected]>
Co-authored-by: tarat44 <[email protected]>

* Update go.mod to pull in latest kubernetes_platform package

Signed-off-by: droctothorpe <[email protected]>
Co-authored-by: edmondop <[email protected]>
Co-authored-by: tarat44 <[email protected]>

---------

Signed-off-by: droctothorpe <[email protected]>
Co-authored-by: edmondop <[email protected]>
Co-authored-by: tarat44 <[email protected]>
PiperOrigin-RevId: 609757226
… ConfigMaps as volumes and as env variables (kubeflow#10483)

* Update kfp backend and kubernetes sdk to support ConfigMaps as volumes and as env

Signed-off-by: Alexey Roytman <[email protected]>

* update go.mod, apiserver.csv and driver.csv

Signed-off-by: Alexey Roytman <[email protected]>

* add test/snapshot/data files

Signed-off-by: Alexey Roytman <[email protected]>

* fix tests

Signed-off-by: Alexey Roytman <[email protected]>

* go mod tidy

Signed-off-by: Alexey Roytman <[email protected]>

* update backend/third_party_licenses/apiserver.csv

Signed-off-by: Alexey Roytman <[email protected]>

* update backend/third_party_licenses/apiserver.csv

Signed-off-by: Alexey Roytman <[email protected]>

* fix comments

Signed-off-by: Alexey Roytman <[email protected]>

* fix comments

Signed-off-by: Alexey Roytman <[email protected]>

* update go.mod, apiserver.csv and driver.csv

Signed-off-by: Alexey Roytman <[email protected]>

---------

Signed-off-by: Alexey Roytman <[email protected]>
…ng. (kubeflow#10494)

* fix(sdk): Prevents dsl.ParallelFor over single paramter from compiling.

* fix(sdk): Prevents dsl.ParallelFor over single paramter from compiling.

* update PR number in release notes

* formatting

* Add compiler_test.py test for single param compile failure

* Update some docstrings and add todo

* formatting

* Update sdk/python/kfp/compiler/compiler_test.py

Co-authored-by: Connor McCarthy <[email protected]>

* Update sdk/python/kfp/compiler/compiler_test.py

Co-authored-by: Connor McCarthy <[email protected]>

* Update sdk/python/kfp/dsl/for_loop.py

Co-authored-by: Connor McCarthy <[email protected]>

* Use print_and_return and other small changes

* typo

* typo

---------

Co-authored-by: Connor McCarthy <[email protected]>
…ter RLHF tuning

PiperOrigin-RevId: 610918020
…n Eval pipeline and components to e2-standard-4

PiperOrigin-RevId: 612531671
…low#10501)

* Updated legal info due to migration from CLA to DCO

Signed-off-by: hbelmiro <[email protected]>

* Updated DCO link

Signed-off-by: hbelmiro <[email protected]>

---------

Signed-off-by: hbelmiro <[email protected]>
…pec (kubeflow#10481)

* Add backend and sdk support for pod spec timeout

Signed-off-by: Tommy Li <[email protected]>

* fix conflicts

Signed-off-by: Tommy Li <[email protected]>

---------

Signed-off-by: Tommy Li <[email protected]>
… the classes which are not predicted in bp outputs

PiperOrigin-RevId: 612920651
… ImagePullPolicy (kubeflow#10417)

* feat(Backend + SDK): Update kfp backend and kubernetes sdk to support ImagePullPolicy.

Signed-off-by: Revital Sur <[email protected]>

* Fix format.

Signed-off-by: Revital Sur <[email protected]>

* Update apiserver.csv.

Signed-off-by: Revital Sur <[email protected]>

* Update licenses.

Signed-off-by: Revital Sur <[email protected]>

---------

Signed-off-by: Revital Sur <[email protected]>
… training. With the train time eval dataset available, the pipeline outputs the accuracy and cross entropy metrics to the log

PiperOrigin-RevId: 613057150
dependabot bot and others added 13 commits April 5, 2024 05:19
…ubeflow#10575)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.6.1 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.6.1...v1.15.6)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ver/visualization (kubeflow#10658)

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.0.1 to 10.3.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](python-pillow/Pillow@10.0.1...10.3.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…esolve_upload_location from function_based

PiperOrigin-RevId: 622081254
…w#10639)

Bumps [express](https://github.com/expressjs/express) from 4.18.2 to 4.19.2.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](expressjs/express@4.18.2...4.19.2)

---
updated-dependencies:
- dependency-name: express
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… `chat` variants of `bison@001` with the `preview.llm.rlhf_pipeline`

PiperOrigin-RevId: 622229648
…/server (kubeflow#10574)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.5.10 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.5.10...v1.15.6)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…low#10659)

these samples currently only work on the Google Cloud distribution
because of hardcoded secret name that only pre-exists there.
1 extract the hardcoded secret name to a const to make it a little
easier to change
2 add a note about it.

Signed-off-by: Greg Sheremeta <[email protected]>
* remove deprecated ibm components

Signed-off-by: tomcli <[email protected]>

* update linkage to the ibm components

Signed-off-by: tomcli <[email protected]>

---------

Signed-off-by: tomcli <[email protected]>
…#10605)

* feat(sdk+backend): Add add_ephemeral_volume method to python sdk + add support to backend
Signed-off-by: abaland <[email protected]>

* feat(sdk+backend): Add add_ephemeral_volume method to python sdk + add support to backend

Signed-off-by: abaland <[email protected]>

* chore: upgrade go module + go mod tidy

Signed-off-by: abaland <[email protected]>

* chore: upgrade license files

Signed-off-by: abaland <[email protected]>

---------

Signed-off-by: abaland <[email protected]>
…ith the `preview.llm.rlhf_pipeline`

PiperOrigin-RevId: 623336791
@gmfrasca gmfrasca force-pushed the argo-3416 branch 2 times, most recently from 57a43e0 to 80179cb Compare April 11, 2024 20:38
connor-mccarthy and others added 13 commits April 11, 2024 23:31
PiperOrigin-RevId: 625114315
* Merge kfp-tekton backend code

Signed-off-by: Ricardo M. Oliveira <[email protected]>

* Add swf work

Signed-off-by: Ricardo M. Oliveira <[email protected]>

---------

Signed-off-by: Ricardo M. Oliveira <[email protected]>
Signed-off-by: Giulio Frasca <[email protected]>
Signed-off-by: Giulio Frasca <[email protected]>
Signed-off-by: Giulio Frasca <[email protected]>
- Argo 3.4.16 upgrade introduces a breaking change with inconsistent node.ID vs
  node.Name
- introduce a function in workflow.go to conditionally handle this

Signed-off-by: Giulio Frasca <[email protected]>
- PNS Executor was removed in Argo v3.4, so manifests no longer valid
- WorkflowController will fail to start if `containerRuntimeExecutor`
  provided as input parameter, so remove from WC ConfigMap and CM
  patches

Signed-off-by: Giulio Frasca <[email protected]>
- Stemming from upgrade to argo 3.4, Pod Name is no longer always the
  same as NodeID, which breaks a few tabs (PodInfo, PodEvents and
  PodLogs).  Add function to address this

Signed-off-by: Giulio Frasca <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.