Deploy on Google Cloud Platform

This tutorial provides an overview of the steps involved in setting up a CRE on Google cloud platform using command-line.

Note: Please follow Starting a PhenoMeNal CRE on a public or private cloud provider for the general prerequisites for a deployment on a private of public cloud provider.

Google cloud specific prerequisites

• You have enabled the Google Compute Engine API: API Manager > Library > Compute Engine API > Enable (detailed instructions)

• You have created and downloaded a service account file for your GCE project: Api manager > Credentials > Create credentials > Service account key (detailed instructions)

Configuration

All of the commands in this documentation are meant to be run in the config directory created by the command below.

Start by creating a configuration directory:

kn --preset phenomenal init gce my-vre-config-dir
cd my-vre-config-dir

Inside this configuration directory you will need to edit the file config.tfvars where you will need to set:

Cluster

• cluster_prefix: every resource in your tenancy will be named with this prefix

Google credentials

• gce_credentials_file: You need to save your GCE service account file as service-account.json in config-dir
• gce_project: your project id (e.g. strange-animal-111222) - Can be found in your GCE service account file
• gce_zone: Some GCE zone for your deployment (e.g. europe-west1-b)

Master configuration

• master_flavor: an instance flavor for the master
• master_disk_size: Size in GB, default 20 is enough for normal deployments
• master_as_edge: master is acting as gateway for accessing services (leave true if you are unsure)

Node configuration

• node_count: number of Kubernetes nodes to be created (no floating IP is needed for these nodes)
• node_flavor: an instance flavor name for the Kubernetes nodes
• node_disk_size: Size in GB, default 20 is enough for normal deployments

Gluster configuration - See: KubeNow Gluster documentation.

• glusternode_count: number of egde nodes to be created (1 - 3 depending on preferred replication factor)
• glusternode_flavor: an instance flavor for the glusternodes
• glusternode_disk_size: Size in GB, default 20 is enough for normal deployments
• glusternode_extra_disk_size: disk size of the fileserver disk in GB (depending on the size of your dataset)

Edge configuration (optional) - See: KubeNow Edge documentation.

• edge_count: number of egde nodes to be created
• edge_flavor: an instance flavor for the edge nodes
• edge_disk_size: Size in GB, default 20 is enough for normal deployments

Cloudflare (optional) - See: KubeNow Cloudflare documentation.

• use_cloudflare: wether you want to use cloudflare as dns provider
• cloudflare_email: the mail that you used to register your Cloudflare account
• cloudflare_token: an authentication token that you can generate from the Cloudflare web interface
• cloudflare_domain: a zone that you created in your Cloudflare account. This typically matches your domain name (e.g. somedomain.com)
• cloudflare_subdomain: "a subdomain for this deployment"

Cloudflare proxy (optional) - See: KubeNow Cloudflare proxy documentation.

• cloudflare_proxied:: to proxy or not, e.g. true
• cloudflare_record_texts: name on services to be proxied

In the provision sub-section of the config.tfvars config file you can edit the following parameters: (Also see KubeNow Provisioning documentation.)

Services

• password_all_services: password for all your services (e.g. Galaxy, Jupyter etc.)
• username_all_services: username for all your services (e.g. Galaxy, Jupyter etc.)

Galaxy

• galaxy_include: should service be deployed in cluster (true/false)
• galaxy_admin_email: the local galaxy admin (you?)

Jupyter

• jupyter_include: should service be deployed in cluster (true/false)

Luigi

• luigi_include: should service be deployed in cluster (true/false)

Kubernetes dashboard

• dashboard_include: should service be deployed in cluster (true/false)

Logging and monitoring services - See: Logging and monitoring wiki

• logmon_include: should service be deployed in cluster (true/false)

Pachyderm + Minio (optional) - See: Pachyderm tutorial with MTBLS data

• pachyderm_release_name: a release name for the Pachyderm service
• pachyderm_etcd_pvc_size: storage dedicated for etcd (In GB)
• minio_release_name: release name for the Minio service
• minio_pvc_size: storage dedicated for the Minio service (In GB)
• minio_accesskey: access key for the S3 endpoint
• minio_secretkey: secret key for the S3 endpoint
• minio_replicas: number of replicas of the Minio service

Once you are done with your settings you are ready to deploy the cluster:

kn apply

when deployment is finished then you should be able to reach the services at:

Galaxy         = http://galaxy.<your-prefix>.<yourdomain>
Jupyter        = http://notebook.<your-prefix>.<yourdomain>
Luigi          = http://luigi.<your-prefix>.<yourdomain>
Kube-dashboard = http://dashboard.<your-prefix>.<yourdomain>
Pachyderm      = ssh into the master node and use pachctl. Pachyderm tutorial: https://github.com/phnmnl/MTBLS233-Pachyderm

and if you want to ssh into the master node:

kn ssh

and to destroy use:

kn destroy

PhenoMeNal help and support

For feedback and help