Skip to content

Commit

Permalink
Merge branch 'GoogleCloudPlatform:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
@moficodes
moficodes committed Apr 2, 2024
2 parents 97598ac + 78f9ba2 commit 96e54be
Show file tree
Hide file tree
Showing 555 changed files with 18,926 additions and 3,044 deletions.
23 changes: 19 additions & 4 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,25 @@
# See the License for the specific language governing permissions and
# limitations under the License.

terraform.tfvars
terraform.tfstate*
.terraform*
__pycache__/
## Archives
**/*.tar
**/*.tar.gz
**/*.zip

# Directories
bin/
deploy/

# IDEs
.idea/

# Python
__pycache__/

# Terraform
default.tfstate
default.tfstate.backup
.terraform*
terraform.tfstate*
terraform.tfvars
tfplan
30 changes: 14 additions & 16 deletions applications/jupyter/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ This code can also perform auto brand creation. Please check the details [below]
* Terraform
* Gcloud CLI

Jupyterhub server can use either local storage or GCS to store notebooks and other artifcts.
JupyterHub server can use either local storage or GCS to store notebooks and other artifcts.
To use GCS, create a bucket with your username. For example, when authenticating with IAP as [email protected], ensure your bucket name is `gcsfuse-<username>`

## Installation
Expand All @@ -43,7 +43,7 @@ To use GCS, create a bucket with your username. For example, when authenticating
cd ai-on-gke/applications/jupyter
```

2. Edit `workloads.tfvars` with your GCP settings. The `namespace` that you specify will become a K8s namespace for your Jupyterhub services. For more information about what the variables do visit [here](https://github.com/GoogleCloudPlatform/ai-on-gke/blob/main/applications/jupyter/variable_definitions.md)
2. Edit `workloads.tfvars` with your GCP settings. The `namespace` that you specify will become a K8s namespace for your JupyterHub services. For more information about what the variables do visit [here](https://github.com/GoogleCloudPlatform/ai-on-gke/blob/main/applications/jupyter/variable_definitions.md)

**Important Note:**
If using this with the Ray module (`applications/ray/`), it is recommended to use the same k8s namespace
Expand All @@ -55,12 +55,12 @@ for both i.e. set this to the same namespace as `applications/ray/workloads.tfva
| cluster_name | GKE Cluster Name | Yes |
| cluster_location | GCP Region | Yes |
| cluster_membership_id | Fleet membership name for GKE cluster. <br /> Required when using private clusters with Anthos Connect Gateway | |
| namespace | The namespace that Jupyterhub and rest of the other resources will be installed in. | Yes |
| namespace | The namespace that JupyterHub and rest of the other resources will be installed in. | Yes |
| gcs_bucket | GCS bucket to be used for Jupyter storage | |
| create_service_account | Create service accounts used for Workload Identity mapping | Yes |
| gcp_and_k8s_service_account | GCP service account used for Workload Identity mapping and k8s sa attached with workload | Yes |

For variables under `Jupyterhub with IAP`, please see the section below
For variables under `JupyterHub with IAP`, please see the section below

### Secure endpoint with IAP

Expand All @@ -78,7 +78,7 @@ See the example `.tfvars` files under `/applications/jupyter` for different bran

| Variable | Description | Default Value | Required |
| ------------------------ |--------------------------- |:-------------:|:--------:|
| add_auth | Enable IAP on Jupyterhub | true | Yes |
| add_auth | Enable IAP on JupyterHub | true | Yes |
| brand | Name of the brand used for creating IAP OAuth clients. Only one is allowed per project. View existing brands: `gcloud iap oauth-brands list`. Leave it empty to create a new brand. Uses [support_email](#support_email) | | |
| support_email | Support email assocated with the [brand](#brand). Used as a point of contact for consent for the ["OAuth Consent" in Cloud Console](https://console.cloud.google.com/apis/credentials/consent). Optional field if `brand` is empty. | | |
| default_backend_service | default_backend_service | | |
Expand All @@ -87,7 +87,7 @@ See the example `.tfvars` files under `/applications/jupyter` for different bran
| url_domain_name | This variable will only be used if [url_domain_addr](#url_domain_addr) is provided. It is the name associated with the domain provided by the user. Since we are using Ingress, it will require the `kubernetes.io/ingress.global-static-ip-name` annotation along with the name associated. | | |
| client_id | Client ID of an [OAuth 2.0 Client ID](https://console.cloud.google.com/apis/credentials) created by the user for enabling IAP. You must also input the [client_secret](#client_secret). If this variable is unset, the template will create an OAuth client for you - in this case, you must ensure the associated [brand](https://console.cloud.google.com/apis/credentials/consent) is `Internal` i.e. only principals within the organization can access the application. | | |
| client_secret | Client Secret associated with the [client_id](#client_id). This variable will only be used when the client id is filled out. | | |
| members_allowlist | Comma seperated values for users to be allowed access through IAP. Example values: `allAuthenticatedUsers` or `allAuthenticatedUsers,user:[email protected]` | allAuthenticatedUsers | |
| members_allowlist | Comma seperated values for users to be allowed access through IAP. Example values: `user:[email protected]` | | |


### Install
Expand All @@ -109,19 +109,17 @@ gcloud auth application-default login
- Should have `jupyter-proxy-public` in the name eg.: `k8s1-63da503a-jupyter-proxy-public-80-74043627`.
* Run `terraform apply --var-file=./workloads.tfvars`

## Using Jupyterhub
## Using JupyterHub

### If Auth with IAP is disabled

1. Extract the randomly generated password for Jupyterhub login
1. Extract the randomly generated password for JupyterHub login

```
terraform output password
```

2. Visit [Services](https://console.cloud.google.com/kubernetes/discovery) section on the GKE console & open the external IP for the `proxy-public` service in the browser.

> **_NOTE:_** If there isn't an external IP for `proxy-public`, is it most likely due to authentication being enabled.
2. Setup port forwarding for the frontend: `kubectl port-forward service/proxy-public -n <namespace> 8081:80 &`, and open `localhost:8081` in a browser.

### If Auth with IAP is enabled

Expand All @@ -139,17 +137,17 @@ Please note there may be some propagation delay after adding IAP principals (5-1
### Setup Access

In order for users to login to Jupyterhub via IAP, their access needs to be configured. To allow access for users/groups:
In order for users to login to JupyterHub via IAP, their access needs to be configured. To allow access for users/groups:

1. Navigate to the [GCP IAP Cloud Console](https://console.cloud.google.com/security/iap) and select your backend-service for `<namespace>/proxy-public`.

2. Click on `Add Principal`, insert the username / group name and select under `Cloud IAP` with role `IAP-secured Web App User`. Once presmission is granted, these users / groups can login to Jupyterhub with IAP. Please note there may be some propagation delay after adding IAP principals (5-10 mins).
2. Click on `Add Principal`, insert the username / group name and select under `Cloud IAP` with role `IAP-secured Web App User`. Once presmission is granted, these users / groups can login to JupyterHub with IAP. Please note there may be some propagation delay after adding IAP principals (5-10 mins).

## Persistent Storage

Jupyterhub is configured to provide 2 choices for storage:
JupyterHub is configured to provide 2 choices for storage:

1. Default Jupyterhub Storage - `pd.csi.storage.gke.io` with reclaim policy `Delete`
1. Default JupyterHub Storage - `pd.csi.storage.gke.io` with reclaim policy `Delete`

2. GCSFuse - `gcsfuse.csi.storage.gke.io` uses GCS Buckets and require users to pre-create buckets with name format `gcsfuse-{username}`

Expand Down Expand Up @@ -194,4 +192,4 @@ This module uses `<ip>.nip.io` as the domain name with a global static ipv4 addr

## Additional Information

For more information about Jupyterhub profiles and the preset profiles visit [here](https://github.com/GoogleCloudPlatform/ai-on-gke/blob/main/applications/jupyter/profiles.md)
For more information about JupyterHub profiles and the preset profiles visit [here](https://github.com/GoogleCloudPlatform/ai-on-gke/blob/main/applications/jupyter/profiles.md)
64 changes: 45 additions & 19 deletions applications/jupyter/main.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,36 @@ data "google_project" "project" {
project_id = var.project_id
}

## Enable Required GCP Project Services APIs
module "project-services" {
source = "terraform-google-modules/project-factory/google//modules/project_services"
version = "~> 14.5"

project_id = var.project_id
disable_services_on_destroy = false
disable_dependent_services = false
activate_apis = flatten([
"autoscaling.googleapis.com",
"cloudbuild.googleapis.com",
"cloudresourcemanager.googleapis.com",
"compute.googleapis.com",
"config.googleapis.com",
"connectgateway.googleapis.com",
"container.googleapis.com",
"containerfilesystem.googleapis.com",
"dns.googleapis.com",
"gkehub.googleapis.com",
"iamcredentials.googleapis.com",
"logging.googleapis.com",
"monitoring.googleapis.com",
"pubsub.googleapis.com",
"servicenetworking.googleapis.com",
"serviceusage.googleapis.com",
"sourcerepo.googleapis.com",
(var.add_auth ? ["iap.googleapis.com"] : [])
])
}

module "infra" {
source = "../../infrastructure"
count = var.create_cluster ? 1 : 0
Expand All @@ -39,12 +69,14 @@ module "infra" {
subnetwork_name = "default"
cpu_pools = var.cpu_pools
enable_gpu = false
depends_on = [module.project-services]
}

data "google_container_cluster" "default" {
count = var.create_cluster ? 0 : 1
name = var.cluster_name
location = var.cluster_location
count = var.create_cluster ? 0 : 1
name = var.cluster_name
location = var.cluster_location
depends_on = [module.project-services]
}

locals {
Expand All @@ -54,10 +86,12 @@ locals {
cluster_membership_id = var.cluster_membership_id == "" ? var.cluster_name : var.cluster_membership_id
enable_autopilot = var.create_cluster ? var.autopilot_cluster : data.google_container_cluster.default[0].enable_autopilot
host = local.private_cluster ? "https://connectgateway.googleapis.com/v1/projects/${data.google_project.project.number}/locations/${var.cluster_location}/gkeMemberships/${local.cluster_membership_id}" : local.endpoint
kubernetes_namespace = var.goog_cm_deployment_name != "" ? "${var.goog_cm_deployment_name}-${var.kubernetes_namespace}" : var.kubernetes_namespace
}

locals {
workload_identity_service_account = var.goog_cm_deployment_name != "" ? "${var.goog_cm_deployment_name}-${var.workload_identity_service_account}" : var.workload_identity_service_account
jupyterhub_default_uri = "https://console.cloud.google.com/kubernetes/service/${var.cluster_location}/${var.cluster_name}/${local.kubernetes_namespace}/proxy-public/overview?project=${var.project_id}"
}

provider "kubernetes" {
Expand Down Expand Up @@ -101,33 +135,26 @@ module "gcs" {
module "namespace" {
source = "../../modules/kubernetes-namespace"
providers = { helm = helm.jupyter }
namespace = var.kubernetes_namespace
namespace = local.kubernetes_namespace
create_namespace = true
}

# IAP Section: Enabled the IAP service
resource "google_project_service" "project_service" {
count = var.add_auth ? 1 : 0
project = var.project_id
service = "iap.googleapis.com"

disable_dependent_services = false
disable_on_destroy = false
}

# Creates jupyterhub
module "jupyterhub" {
source = "../../modules/jupyter"
providers = { helm = helm.jupyter, kubernetes = kubernetes.jupyter }
project_id = var.project_id
namespace = var.kubernetes_namespace
namespace = local.kubernetes_namespace
additional_labels = var.additional_labels
workload_identity_service_account = local.workload_identity_service_account
gcs_bucket = var.gcs_bucket
autopilot_cluster = local.enable_autopilot
notebook_image = "jupyter/tensorflow-notebook"
notebook_image_tag = "python-3.10"

# IAP Auth parameters
add_auth = var.add_auth
brand = var.brand
create_brand = var.create_brand
support_email = var.support_email
client_id = var.client_id
client_secret = var.client_secret
Expand All @@ -137,8 +164,7 @@ module "jupyterhub" {
k8s_backend_config_name = var.k8s_backend_config_name
k8s_backend_service_name = var.k8s_backend_service_name
k8s_backend_service_port = var.k8s_backend_service_port
url_domain_addr = var.url_domain_addr
url_domain_name = var.url_domain_name
members_allowlist = var.members_allowlist
domain = var.domain
members_allowlist = var.members_allowlist != "" ? split(",", var.members_allowlist) : []
depends_on = [module.gcs, module.namespace]
}
Loading

0 comments on commit 96e54be

Please sign in to comment.