From a465e72db8363efc8c58573e5a662ef0677edd89 Mon Sep 17 00:00:00 2001 From: Mohamed Khairallah Gharbi Date: Mon, 25 Nov 2024 16:58:18 +0100 Subject: [PATCH] add gcp documentation --- .docs/content/1.installation/3.aws/_dir.yml | 2 +- .../4.gcp/0. gcp-all-in-one-deployment.md | 79 +++++++++ .../4.gcp/1. gcp-troubleshooting.md | 74 +++++++++ .docs/content/1.installation/4.gcp/_dir.yml | 2 +- .../4.gcp/gcp-all-in-one-deployment.md | 153 ------------------ 5 files changed, 155 insertions(+), 155 deletions(-) create mode 100644 .docs/content/1.installation/4.gcp/0. gcp-all-in-one-deployment.md create mode 100644 .docs/content/1.installation/4.gcp/1. gcp-troubleshooting.md delete mode 100644 .docs/content/1.installation/4.gcp/gcp-all-in-one-deployment.md diff --git a/.docs/content/1.installation/3.aws/_dir.yml b/.docs/content/1.installation/3.aws/_dir.yml index f344c96cf..6954dbbb2 100644 --- a/.docs/content/1.installation/3.aws/_dir.yml +++ b/.docs/content/1.installation/3.aws/_dir.yml @@ -1,2 +1,2 @@ title: AWS -navigation.icon: vscode-icons:file-type-aws +navigation.icon: vscode-icons:folder-type-aws diff --git a/.docs/content/1.installation/4.gcp/0. gcp-all-in-one-deployment.md b/.docs/content/1.installation/4.gcp/0. gcp-all-in-one-deployment.md new file mode 100644 index 000000000..e2a95f166 --- /dev/null +++ b/.docs/content/1.installation/4.gcp/0. gcp-all-in-one-deployment.md @@ -0,0 +1,79 @@ +# GCP all in one deployment Guide + +This guide will help you deploy your Armonik project on Google Cloud Platform (GCP). + +## Step 1: Preparation + +#### 1. Install Google CLI + +When receiving your credentials (email and password). +Download and install the Google CLI by following the instructions on the [following link](https://cloud.google.com/sdk/docs/install?hl=fr#deb) + +#### 2. Initial Setup + +After installation, authenticate using the provided credentials (email and password) and select the project you want to deploy the resources to. +Follow [following tutorial](https://cloud.google.com/docs/authentication/provide-credentials-adc?hl=fr#how-to) to authenticate into the CLI. + +You can authenticate using the following command: + +```bash +gcloud auth login +``` + +To configure the project, if you don't know the project ID, you can list all the projects using the following command: + +```bash +gcloud projects list +``` + +To configure the project, use the following command: + +```bash +gcloud config set project +``` + +Once authenticated, you should see a confirmation page with the message: + +*You are now authenticated with the gcloud CLI!* + +## Step 2: Deployment +Navigate to the **gcp folder** (infrastructure/quick-deploy/gcp) and follow the steps below to deploy your resources. +### 1. Boostrap + +Generate a prefix key ready in the GCP environment to deploy your resources. + +```bash +make bootstrap-deploy PREFIX= +``` + +### 2. Deploy + +To deploy your resources, execute: + +```bash +make deploy PREFIX= +``` + +## Step 3: Cleanup + +### 1. Destroy the deployment + +#### Attention: After using the deployment, you have to make sure to destroy it to avoid any additional costs. #### + +To destroy the deployment, use the following command: + +```bash +make destroy PREFIX= +``` + +### 2. Destroy the GCP Prefix Key + +To clean up the GCP prefix key, use the following command: + +```bash +make bootstrap-destroy PREFIX= +``` + +## Step 4: Add a Sample Partition + +Just like with an **AWS** or **localhost** deployment, you can add a sample partition to test deployment on the **GCP** environment. You need to build the images and redeploy the services after adding the sample partition in the parameters.tfvars file. diff --git a/.docs/content/1.installation/4.gcp/1. gcp-troubleshooting.md b/.docs/content/1.installation/4.gcp/1. gcp-troubleshooting.md new file mode 100644 index 000000000..91a4f6fde --- /dev/null +++ b/.docs/content/1.installation/4.gcp/1. gcp-troubleshooting.md @@ -0,0 +1,74 @@ +# GCP Troubleshooting Guide + +This guide will help you troubleshoot common issues when deploying your Armonik project on Google Cloud Platform (GCP). + +## 1. Deployment Error with New Partition +If you encounter an error during deployment after adding a new partition with the value `replicas > 0`, follow these steps: +1. Set the replicas to `0`. +2. Redeploy the resources. +3. If the deployment succeeds, update the replicas to the desired value. +4. Redeploy the resources again. + +--- + +## 2. Pub/Sub Client-Side Issue +If you encounter an issue with the Pub/Sub client and see the error message: `FAILED_PRECONDITION: Requested entity was not found.`, follow the steps below to resolve the issue. + +### Error Message +When running the application, you may encounter the following error: + +```bash +grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: + status = StatusCode.FAILED_PRECONDITION + details = "Cloud Pub/Sub did not have the necessary permissions configured to support this operation. + Please verify that the service account [SERVICE_ACCOUNT] was granted the Cloud KMS CryptoKey Encrypter/Decrypter role + for the project containing the CryptoKey resource [PROJECT_ID]/locations/[LOCATION]/keyRings/[KEY_RING]/cryptoKeys/[CRYPTO_KEY]." + debug_error_string = "UNKNOWN:Error received from peer {created_time:"[TIMESTAMP]", + grpc_status:9, + grpc_message:"Cloud Pub/Sub did not have the necessary permissions configured to support this operation. + Please verify that the service account [SERVICE_ACCOUNT] was granted the Cloud KMS CryptoKey Encrypter/Decrypter role + for the project containing the CryptoKey resource [PROJECT_ID]/locations/[LOCATION]/keyRings/[KEY_RING]/cryptoKeys/[CRYPTO_KEY]."} +``` + +### Problem Description +This error occurs because Cloud Pub/Sub requires access to the specified CMEK to encrypt or decrypt messages. The service account used by Pub/Sub does not have the **Cloud KMS CryptoKey Encrypter/Decrypter** role for the specified CryptoKey. + +### Key Components in the Error: +- **Service Account**: `service-[NUMERIC_ID]@gcp-sa-pubsub.iam.gserviceaccount.com` +- **CryptoKey Resource**: + - Project: `[PROJECT_ID]` + - Location: `[LOCATION]` (e.g., `europe-west1`) + - Key Ring: `[KEY_RING]` + - CryptoKey: `[CRYPTO_KEY]` +- **Missing Role**: `roles/cloudkms.cryptoKeyEncrypterDecrypter` + +Without this role, Cloud Pub/Sub cannot perform encryption or decryption using the CMEK. + +### Solution + +To resolve the issue, grant the **Cloud KMS CryptoKey Encrypter/Decrypter** role to the Pub/Sub service account for the specified CryptoKey. + +#### Step 1: Identify the Service Account +The service account mentioned in the error typically has the format: `service-[NUMERIC_ID]@gcp-sa-pubsub.iam.gserviceaccount.com`. This service account is automatically created by Google Cloud to manage Pub/Sub operations. + +#### Step 2: Grant the Necessary Role +You can grant the **Cloud KMS CryptoKey Encrypter/Decrypter** role to the service account using the Google Cloud Console or the gcloud command-line tool. + +##### Using the Google Cloud Console +1. Open the [Google Cloud Console](https://console.cloud.google.com). +2. Navigate to **Key Management > CryptoKeys**. +3. Locate the CryptoKey resource: + - **Project**: `[PROJECT_ID]` + - **Location**: `[LOCATION]` (e.g., `europe-west1`) + - **Key Ring**: `[KEY_RING]` + - **CryptoKey**: `[CRYPTO_KEY]` +4. Click on the CryptoKey and go to the **Permissions** tab. +5. Add the service account as a principal: + - **Principal**: `service-[NUMERIC_ID]@gcp-sa-pubsub.iam.gserviceaccount.com` + - **Role**: `Cloud KMS CryptoKey Encrypter/Decrypter` +6. Save the changes. + +--- + +After granting the role, Cloud Pub/Sub should be able to access the specified CMEK for encryption and decryption operations. Retry the operation that triggered the error. Ensure that the deployment is successful and verify that the Pub/Sub client can now access the CMEK without any issues. + diff --git a/.docs/content/1.installation/4.gcp/_dir.yml b/.docs/content/1.installation/4.gcp/_dir.yml index 0dc1d33df..fa8498148 100644 --- a/.docs/content/1.installation/4.gcp/_dir.yml +++ b/.docs/content/1.installation/4.gcp/_dir.yml @@ -1,2 +1,2 @@ title: GCP -navigation.icon: vscode-icons:file-type-gcp +navigation.icon: vscode-icons:folder-type-gcp diff --git a/.docs/content/1.installation/4.gcp/gcp-all-in-one-deployment.md b/.docs/content/1.installation/4.gcp/gcp-all-in-one-deployment.md deleted file mode 100644 index 5167c7bc7..000000000 --- a/.docs/content/1.installation/4.gcp/gcp-all-in-one-deployment.md +++ /dev/null @@ -1,153 +0,0 @@ -# GCP all in one deployment Guide - -This guide will help you deploy your Armonik project on Google Cloud Platform (GCP). - - - -## Step 1: Preparation - -### 1. Install Google CLI - -When receiving your credentials (email and password). -Download and install the Google CLI by following the instructions on the [following link](https://cloud.google.com/sdk/docs/install?hl=fr#deb) - - -### 2. Initial Setup - - -After installation, authenticate using the provided credentials (email and password) and select the project **armonik gcp 13469** -Follow [following tutorial](https://cloud.google.com/docs/authentication/provide-credentials-adc?hl=fr#how-to) to authenticate into the CLI. - -You can authenticate using the following command: - -```bash -gcloud auth login -``` - -To configure the project, if you don't know the project ID, you can list all the projects using the following command: - -```bash -gcloud projects list -``` - -To configure the project, use the following command: - -```bash -gcloud config set project -``` - -Once authenticated, you should see a confirmation page with the message: - -*You are now authenticated with the gcloud CLI!* - - -## Step 2: Deployment - -### 1. Boostrap - -Navigate to the **gcp folder** (infrastructure/quick-deploy/gcp) and generate a prefix key by running the following command: - -```bash -make bootstrap-deploy PREFIX= -``` - -### 2. Deploy - -To deploy your resources, execute: -``` -make deploy PREFIX= -``` - -## Step 3: Cleanup - -### 1. Destroy the deployment - -#### After using the deployment, you have to make sure to destroy it to avoid any additional costs. #### - -To destroy the deployment, use the following command: - -```bash -make destroy PREFIX= -``` - -### 2. Destroy the GCP Prefix Key - -To clean up the GCP prefix key, use the following command: - -```bash -make bootstrap-destroy PREFIX= -``` - - -## Step 4: Add a Sample Partition - -Just like with an **AWS** or **localhost** deployment, you can add a sample partition to test deployment on the **GCP** environment. - - -## Troubleshooting - -### 1. Deployment Error with New Partition -In case of an error during deployment after adding a new partition with the value replicas > 0, you can put the replicas to 0 and redeploy the resources. -If the deployment succeeds, you can then update the replicas to the desired value and redeploy the resources. - ---- -### 2. Pub/Sub Issue client side -If you encounter an issue with the Pub/Sub client, with the following error message: `FAILED_PRECONDITION: Requested entity was not found.`, you can follow the steps in the [following link](https://cloud.google.com/pubsub/docs/customer-managed-encryption-keys#troubleshooting) to resolve the issue. - ---- -#### Error Message - -When running the application, you may encounter the following error: -grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.FAILED_PRECONDITION details = "Cloud Pub/Sub did not have the necessary permissions configured to support this operation. Please verify that the service account [SERVICE_ACCOUNT] was granted the Cloud KMS CryptoKey Encrypter/Decrypter role for the project containing the CryptoKey resource [PROJECT_ID]/locations/[LOCATION]/keyRings/[KEY_RING]/cryptoKeys/[CRYPTO_KEY]." debug_error_string = "UNKNOWN:Error received from peer {created_time:"[TIMESTAMP]", grpc_status:9, grpc_message:"Cloud Pub/Sub did not have the necessary permissions configured to support this operation. Please verify that the service account [SERVICE_ACCOUNT] was granted the Cloud KMS CryptoKey Encrypter/Decrypter role for the project containing the CryptoKey resource [PROJECT_ID]/locations/[LOCATION]/keyRings/[KEY_RING]/cryptoKeys/[CRYPTO_KEY]."}" - - ---- - -#### Problem Description - -This error occurs because Cloud Pub/Sub requires access to the specified CMEK to encrypt or decrypt messages. The service account used by Pub/Sub does not have the **Cloud KMS CryptoKey Encrypter/Decrypter** role for the specified CryptoKey. - -#### Key Components in the Error: -- **Service Account**: `service-[NUMERIC_ID]@gcp-sa-pubsub.iam.gserviceaccount.com` -- **CryptoKey Resource**: - - Project: `[PROJECT_ID]` - - Location: `[LOCATION]` (e.g., `europe-west1`) - - Key Ring: `[KEY_RING]` - - CryptoKey: `[CRYPTO_KEY]` -- **Missing Role**: `roles/cloudkms.cryptoKeyEncrypterDecrypter` - -Without this role, Cloud Pub/Sub cannot perform encryption or decryption using the CMEK. - ---- - -## Solution - -To resolve the issue, grant the **Cloud KMS CryptoKey Encrypter/Decrypter** role to the Pub/Sub service account for the specified CryptoKey. - -### Step 1: Identify the Service Account -The service account mentioned in the error typically has the format: service-[NUMERIC_ID]@gcp-sa-pubsub.iam.gserviceaccount.com - -This service account is automatically created by Google Cloud to manage Pub/Sub operations. ---- -### Step 2: Grant the Necessary Role - -#### ** Using the Google Cloud Console** -1. Open the [Google Cloud Console](https://console.cloud.google.com). -2. Navigate to **Key Management > CryptoKeys**. -3. Locate the CryptoKey resource: - - **Project**: `[PROJECT_ID]` - - **Location**: `[LOCATION]` (e.g., `europe-west1`) - - **Key Ring**: `[KEY_RING]` - - **CryptoKey**: `[CRYPTO_KEY]` -4. Click on the CryptoKey and go to the **Permissions** tab. -5. Add the service account as a principal: - - **Principal**: `service-[NUMERIC_ID]@gcp-sa-pubsub.iam.gserviceaccount.com` - - **Role**: `Cloud KMS CryptoKey Encrypter/Decrypter`. -6. Save the changes. - ---- -After granting the role, Cloud Pub/Sub should be able to access the specified CMEK for encryption and decryption operations. You can now retry the operation that triggered the error. You should deploy the resources again to ensure that the changes take effect. -When the deployment is successful, you can verify that the Pub/Sub client can now access the CMEK without any issues. - - -