Skip to content

Commit

Permalink
Merge branch 'master' into patch-15
Browse files Browse the repository at this point in the history
  • Loading branch information
juliusvonkohout authored Jul 27, 2023
2 parents 1beb8ec + 6de2b20 commit 9b38b6a
Show file tree
Hide file tree
Showing 234 changed files with 51,960 additions and 22,059 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/pipeline_kind_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:

- name: Deploy test pipeline
run: |
pip3 install kfp==1.8.4
pip3 install kfp==1.8.22
kustomize build apps/profiles/upstream/overlays/kubeflow | kubectl apply -f -
kubectl wait --for=condition=Ready pods --all --all-namespaces --timeout 180s
kustomize build common/user-namespace/base | kubectl apply -f -
Expand Down
26 changes: 26 additions & 0 deletions .github/workflows/ray_kind_test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Build & Apply Ray manifest in KinD
on:
pull_request:
paths:
- contrib/ray/**

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Install KinD
run: ./tests/gh-actions/install_kind.sh

- name: Create KinD Cluster
run: kind create cluster --image=kindest/node:v1.23.0

- name: Install kustomize
run: ./tests/gh-actions/install_kustomize.sh

- name: Build & Apply manifests
run: |
cd contrib/ray/
make test
2 changes: 1 addition & 1 deletion OWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ approvers:
- StefanoFioravanzo
- yanniszark
reviewers:
- annajung
- annajung
24 changes: 11 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@ used from the different projects of Kubeflow:

| Component | Local Manifests Path | Upstream Revision |
| - | - | - |
| Istio | common/istio-1-16 | [1.16.0](https://github.com/istio/istio/releases/tag/1.16.0) |
| Istio | common/istio-1-17 | [1.17.3](https://github.com/istio/istio/releases/tag/1.17.3) |
| Knative | common/knative/knative-serving <br /> common/knative/knative-eventing | [1.8.1](https://github.com/knative/serving/releases/tag/knative-v1.8.1) <br /> [1.8.1](https://github.com/knative/eventing/releases/tag/knative-v1.8.1) |
| Cert Manager | common/cert-manager | [1.10.1](https://github.com/cert-manager/cert-manager/releases/tag/v1.10.1) |
| Cert Manager | common/cert-manager | [1.12.2](https://github.com/cert-manager/cert-manager/releases/tag/v1.12.2) |

## Installation

Expand All @@ -84,8 +84,8 @@ The `example` directory contains an example kustomization for the single command

### Prerequisites

- `Kubernetes` (up to `1.25`) with a default [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/)
- `kustomize` [5.0.0](https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.0.0)
- `Kubernetes` (up to `1.26`) with a default [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/)
- `kustomize` [5.0.3](https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.0.3)
- :warning: Kubeflow is not compatible with earlier versions of Kustomize. This is because we need the [`sortOptions`](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/sortoptions/) field, which is only available in Kustomize 5 and onwards https://github.com/kubeflow/manifests/issues/2388.
- `kubectl`

Expand All @@ -94,16 +94,14 @@ The `example` directory contains an example kustomization for the single command

`kubectl apply` commands may fail on the first try. This is inherent in how Kubernetes and `kubectl` work (e.g., CR must be created after CRD becomes ready). The solution is to simply re-run the command until it succeeds. For the single-line command, we have included a bash one-liner to retry the command.

The reason we do `awk '!/well-defined/'` is because there's a regression in Kustomize 5 and a line is printed in stdout and not stderr https://github.com/kubernetes-sigs/kustomize/issues/5039. We'll remove this command once a future patch version of Kustomize is available.

---

### Install with a single command

You can install all Kubeflow official components (residing under `apps`) and all common services (residing under `common`) using the following command:

```sh
while ! kustomize build example | awk '!/well-defined/' | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done
while ! kustomize build example | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done
```

Once, everything is installed successfully, you can access the Kubeflow Central Dashboard [by logging in to your cluster](#connect-to-your-kubeflow-cluster).
Expand Down Expand Up @@ -164,9 +162,9 @@ network authorization and implement routing policies.
Install Istio:

```sh
kustomize build common/istio-1-16/istio-crds/base | kubectl apply -f -
kustomize build common/istio-1-16/istio-namespace/base | kubectl apply -f -
kustomize build common/istio-1-16/istio-install/base | kubectl apply -f -
kustomize build common/istio-1-17/istio-crds/base | kubectl apply -f -
kustomize build common/istio-1-17/istio-namespace/base | kubectl apply -f -
kustomize build common/istio-1-17/istio-install/base | kubectl apply -f -
```

#### Dex
Expand Down Expand Up @@ -195,7 +193,7 @@ Install Knative Serving:

```sh
kustomize build common/knative/knative-serving/overlays/gateways | kubectl apply -f -
kustomize build common/istio-1-16/cluster-local-gateway/base | kubectl apply -f -
kustomize build common/istio-1-17/cluster-local-gateway/base | kubectl apply -f -
```

Optionally, you can install Knative Eventing which can be used for inference request logging:
Expand Down Expand Up @@ -237,15 +235,15 @@ well.
Install istio resources:

```sh
kustomize build common/istio-1-16/kubeflow-istio-resources/base | kubectl apply -f -
kustomize build common/istio-1-17/kubeflow-istio-resources/base | kubectl apply -f -
```

#### Kubeflow Pipelines

Install the [Multi-User Kubeflow Pipelines](https://www.kubeflow.org/docs/components/pipelines/multi-user/) official Kubeflow component:

```sh
kustomize build apps/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user | awk '!/well-defined/' | kubectl apply -f -
kustomize build apps/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user | kubectl apply -f -
```
This installs argo with the safe-to use runasnonroot emissary executor. Please note that the installer is still responsible to analyze the security issues that arise when containers are run with root access and to decide if the kubeflow pipeline main containers are run as runasnonroot. It is strongly recommended that the pipelines main containers are installed and run as runasnonroot and without any special capabilities to mitigate security risks.

Expand Down
15 changes: 15 additions & 0 deletions common/cert-manager/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Cert Manager

## Upgrade Cert Manager Manifests

The manifests for Cert Manager are based off the following:

- [Cert Manager (v1.12.2)](https://github.com/cert-manager/cert-manager/releases/tag/v1.12.2)

1. Download the cert manager yaml with the following commands:

```sh
# No need to install cert-manager-crds.
export CERT_MANAGER_VERSION='1.12.2'
wget -O ./cert-manager/base/cert-manager.yaml "https://github.com/cert-manager/cert-manager/releases/download/v${CERT_MANAGER_VERSION}/cert-manager.yaml"
```
Loading

0 comments on commit 9b38b6a

Please sign in to comment.