Skip to content

cboneti/google-cloud-daos

 
 

Repository files navigation

DAOS on GCP

Distributed Asynchronous Object Storage (DAOS) on Google Cloud Platform (GCP)

This repository contains:

  • Packer scripts used to build DAOS images with Google Cloud Build
  • Terraform modules that can be used to deploy DAOS Server and Client instances
  • Terraform examples that demonstrate how to use the Terraform modules

Directory structure

.
├── docs            Documentation and Cloud Shell tutorials
├── images          Cloud Build config files and Packer templates
│   └── scripts     Scripts that Packer runs to build images
├── terraform       Terraform content
│   ├── examples    Examples that demonstrate how to use the DAOS Terraform modules
│   └── modules     Terraform modules for deploying DAOS server and client instances
└── tools           Tools used by pre-commit

Prerequisites

In order to deploy DAOS on GCP you will need

Deploying DAOS on GCP

Pre-Deployment Steps

The following pre-deployment steps are required

  1. Set defaults for Google Cloud CLI (gcloud)

    Only needs to be done once in your shell (Cloud Shell or local shell).

  2. Enable service APIs and grant permissions

    Enabling APIs and granting service account permissions only needs to be done once for a GCP project.

  3. Create a Packer image in your GCP project

    In order to build DAOS images with Cloud Build your GCP project must contain a Packer image.

    Building the Packer image only needs to be done once for a GCP project.

  4. Build DAOS Server and Client images

    DAOS Server and Client instances are deployed using images that have DAOS pre-installed.

    Therefore, the images need to be built prior to running Terraform to deploy a DAOS cluster.

Click the button below to open a Cloud Shell tutorial which will guide you through the pre-deployment steps listed above. If you lose your Cloud Shell session you can always come back to this README and click the button again.

DAOS on GCP Pre-Deployment

Deploy a DAOS Cluster with Terraform

After completing the pre-deployment steps listed above, you will need to write your own Terraform configuration for your particular use case.

The terraform/modules in this repo can be used in your Terraform configuration to deploy DAOS server and client instances.

The terraform/examples/daos_cluster example serves as both a reference and a quick way to deploy a DAOS cluster.

Click the button below to open a Cloud Shell tutorial that will walk you through using the terraform/examples/daos_cluster example to deploy a DAOS cluster.

DAOS Cluster Example

Deploy a DAOS Cluster with the Google HPC Toolkit

The HPC Toolkit is an open-source software offered by Google Cloud which makes it easy for customers to deploy HPC environments on Google Cloud.

The HPC Toolkit allows customers to deploy turnkey HPC environments (compute, networking, storage, etc) following Google Cloud best-practices, in a repeatable manner. It is designed to be highly customizable and extensible, and intends to address the HPC deployment needs of a broad range of customers.

The HPC Toolkit includes the following community examples which use the Terraform modules in this repository.

HPC Toolkit Community Example Description
DAOS Cluster Use the HPC Toolkit to deploy a standalone DAOS cluster
DAOS Server with Slurm cluster Use the HPC Toolkit to deploy a set of DAOS servers for storage and a Slurm cluster in which the compute nodes are DAOS clients. The example demonstrates how to use DAOS storage in a Slurm job.

If you are just getting started with deploying DAOS on GCP, it is highly recommended to use the HPC Toolkit as it can save you a lot of time as opposed to developing your own Terraform configuration.

Support

Content in the google-cloud-daos repository is licensed under the Apache License Version 2.0 open-source license.

DAOS is being distributed under the BSD-2-Clause-Patent open-source license.

Intel Corporation provides several ways for the users to get technical support:

  1. Community support is available to everybody through Jira and via the DAOS channel for the Google Cloud users on Slack.

    To access Jira, please follow these steps:

    To access the Slack channel for DAOS on Google Cloud, please follow this link https://daos-stack.slack.com/archives/C03GLTLHA59

    This type of support is provided on a best-effort basis, and it does not have any SLA attached.

  2. Commercial L3 support is available on an on-demand basis. Please get in touch with Intel Corporation to obtain more information.

Links

Development

If you are contributing to this repo, see Development

License

Apache License Version 2.0

About

Terraform Script to Deploy DAOS on GCP

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Shell 75.2%
  • HCL 24.5%
  • Python 0.3%