Feat/grpc http gateway generator

CODEOWNERS

@@ -31,10 +31,11 @@
 /tools/apigee-x-trial-provision         @yuriylesyuk @danistrebel
 /tools/decrypt-hybrid-assets            @yuriylesyuk
 /tools/endpoints-oas-importer           @danistrebel
+/tools/grpc-http-gateway-generator      @danistrebel
 /tools/hybrid-quickstart                @danistrebel
 /tools/oas-apigee-mock                  @markjkelly
 /tools/pipeline-linter                  @seymen @danistrebel
 /tools/pipeline-runner                  @seymen @danistrebel
-/tools/sf-dependency-list               @yuriylesyuk
 /tools/proxy-endpoint-unifier           @anaik91
-/tools/target-server-validator          @anaik91
+/tools/sf-dependency-list               @yuriylesyuk
+/tools/target-server-validator          @anaik91

README.md

@@ -92,7 +92,8 @@ Apigee products.
   A tool to unify/split proxy endpoints based on API basepath.
 - [Apigee Target Server Validator](tools/target-server-validator) -
   A tool to validate all targets in Target Servers & Apigee API Proxy Bundles.
-
+- [gRPC to HTTP Gateway Generator](tools/grpc-http-gateway-generator) -
+  Generate gateways to expose gRPC services with HTTP API management.
 ## Labs
 
 This folder contains raw assets used to generate content to teach a particular

tools/grpc-http-gateway-generator/README.md

@@ -0,0 +1,76 @@
+# gRPC to HTTP Gateway Generator
+
+The purpose of this tool is to generate a gRPC to HTTP gateway based on a protocol buffer.
+It leverages the [gRPC Gateway](https://github.com/grpc-ecosystem/grpc-gateway) project to generate the gateway code.
+
+The generated gRPC gateway can then be used to create HTTP adapters for gRPC services and use them in a classical HTTP REST-based API management environment.
+
+![Architecture Overview](./img/architecture.png)
+
+## Prerequisites
+
+The generator assumed you have the protobuf compiler `protoc` installed.
+
+Note that `protoc` is pre-installed on Cloud Shell.
+
+To install `protoc` with `apt`, run the following command:
+
+```sh
+apt install -y protobuf-compiler
+```
+
+## Generate the Gateway Code
+
+The gateway is generated based on a protocol buffer file that  you supply with the `--proto-path` flag:
+
+```sh
+./generate-gateway.sh --proto-path ./examples/currency.proto
+```
+
+Optionally you can also specify the desired output directory with the `--output` flag. If no output directory is specified the output will be generated in the `./generated/gateway` folder.
+
+## Run the Gateway
+
+You can run the gateway locally with the following commands and point to a gRPC server endpoint that runs on localhost port 9090: 
+
+```sh
+(cd generated/gateway && go run main.go --grpc-server-endpoint localhost:9090)
+```
+
+If you preffer to run the gateway as a container use the following commands to build and run the gateway container:
+
+```sh
+(cd generated/gateway && docker build . -t gateway:latest)
+docker run -p 8080:8080 -e GRPC_SERVER_ENDPOINT=localhost:9090 gateway:latest
+```
+
+## Try it out locally
+
+With the gateway running you can call the automatically generated API endpoints of the gateway to consume your gRPC service.
+
+With the gRPC Gateway pointing to an exposed currency service of the [microservice demo](https://github.com/GoogleCloudPlatform/microservices-demo) the currency conversion request would look like this:
+
+```sh
+curl -X POST localhost:8080/hipstershop.CurrencyService/Convert -d '{"from": {"units": 3, "currency_code": "USD", "nanos": 0}, "to_code": "CHF"}'
+```
+
+Whist the code to request supported currencies looks like this:
+
+```sh
+curl -X POST localhost:8080/hipstershop.CurrencyService/GetSupportedCurrencies
+```
+
+Note that the automatically generated API doesn't yet come in a RESTful format. This is one of the aspects that can be changed in an API management layer to improve the overall usability and security of the API.
+
+### Deploy to Cloud Run
+
+Once we've tested the gateway locally we can deploy it to Cloud Run for production use.
+
+```sh
+(cd generated/gateway && gcloud run deploy currency-grpc-gw --source . --allow-unauthenticated --region europe-west1 --project $PROJECT_ID)
+```
+
+## Expose the Cloud Run Service via Apigee
+
+With the Gateway running in Cloud Run we can now expose it via Apigee.
+For detailed instructions on how to expose a cloud run service in Apigee please see [this reference implementation](https://github.com/apigee/devrel/tree/main/references/cloud-run).

tools/grpc-http-gateway-generator/buf.gen.yaml

@@ -0,0 +1,24 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: v1
+plugins:
+  - plugin: go
+    out: gen/go
+    opt:
+      - paths=source_relative
+  - plugin: go-grpc
+    out: gen/go
+    opt:
+      - paths=source_relative

tools/grpc-http-gateway-generator/examples/currency.proto

@@ -0,0 +1,57 @@
+// Copyright 2020 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+package hipstershop;
+
+// -----------------Currency service-----------------
+
+service CurrencyService {
+  rpc GetSupportedCurrencies(Empty) returns (GetSupportedCurrenciesResponse) {}
+  rpc Convert(CurrencyConversionRequest) returns (Money) {}
+}
+
+message Empty {
+}
+
+// Represents an amount of money with its currency type.
+message Money {
+  // The 3-letter currency code defined in ISO 4217.
+  string currency_code = 1;
+
+  // The whole units of the amount.
+  // For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar.
+  int64 units = 2;
+
+  // Number of nano (10^-9) units of the amount.
+  // The value must be between -999,999,999 and +999,999,999 inclusive.
+  // If `units` is positive, `nanos` must be positive or zero.
+  // If `units` is zero, `nanos` can be positive, zero, or negative.
+  // If `units` is negative, `nanos` must be negative or zero.
+  // For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
+  int32 nanos = 3;
+}
+
+message GetSupportedCurrenciesResponse {
+  // The 3-letter currency code defined in ISO 4217.
+  repeated string currency_codes = 1;
+}
+
+message CurrencyConversionRequest {
+  Money from = 1;
+
+  // The 3-letter currency code defined in ISO 4217.
+  string to_code = 2;
+}

tools/grpc-http-gateway-generator/generate-gateway.sh

@@ -0,0 +1,106 @@
+#!/bin/bash
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+set -e
+
+# check if protoc is installed
+if ! command -v protoc &> /dev/null; then
+  echo "[ERROR] protoc is not installed. Please install protoc before running this script."
+  exit 1
+fi
+
+proto_path=""
+out=""
+
+while [[ $# -gt 0 ]]; do
+  case $1 in
+    --proto-path)
+      proto_path="$2"
+      shift 2
+      ;;
+    --out)
+      out="$2"
+      shift 2
+      ;;
+    --*|-*)
+      echo "Unknown option $1"
+      exit 1
+      ;;
+  esac
+done
+
+# check if proto path is provided
+if [ -z "$proto_path" ]; then
+  echo "[ERROR] Proto path is required supply via variable --proto-path"
+  exit 1
+fi
+
+# check if proto path is a file 
+if [ ! -f "$proto_path" ]; then
+  echo "[ERROR] Proto path is not a file"
+  exit 1
+fi
+
+# check if output directory is provided
+if [ -z "$out" ]; then
+  echo "[WARN] Output directory is not provided via variable --out. Using the default value ./generated/gateway"
+  out="$(pwd)/generated/gateway"
+else
+  mkdir -p "$out"
+  out="$(cd "$out"; pwd -P)"
+fi
+
+script_dir="$( cd -- "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"
+
+temp_dir=$(mktemp -d)
+proto_file_name=$(basename "$proto_path")
+cp "$proto_path" "$temp_dir/$proto_file_name"
+
+
+# install tooling dependencies
+pushd "$script_dir/tools" &> /dev/null
+go mod tidy
+go install \
+    github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway \
+    github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2 \
+    google.golang.org/protobuf/cmd/protoc-gen-go \
+    google.golang.org/grpc/cmd/protoc-gen-go-grpc
+PATH="$PATH:$(go env GOPATH)/bin"
+export PATH
+popd &> /dev/null
+
+# Generate the gRPC adapter
+mkdir -p "$out/adapter"
+
+pushd "$temp_dir" &> /dev/null
+protoc -I . \
+  --go_out "$out/adapter" \
+  --go_opt "M$proto_file_name=.;adapter" \
+  --go-grpc_out "$out/adapter" \
+  --go-grpc_opt "M$proto_file_name=.;adapter" \
+  "./$proto_file_name"
+
+protoc -I . --grpc-gateway_out "$out/adapter" \
+  --grpc-gateway_opt "M$proto_file_name=.;adapter" \
+  --grpc-gateway_opt generate_unbound_methods=true \
+  "./$proto_file_name"
+popd &> /dev/null
+
+cp "$script_dir/templates/main.go" "$out/main.go"
+cp "$script_dir/templates/Dockerfile" "$out/Dockerfile"
+
+(cd "$out/adapter" && go mod init adapter &> /dev/null && go mod tidy &> /dev/null)
+(cd "$out" && go mod init gateway &> /dev/null && go mod edit -replace adapter=./adapter && go mod tidy &> /dev/null)

tools/grpc-http-gateway-generator/pipeline.sh

@@ -0,0 +1,90 @@
+#!/bin/bash
+
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -e # exit on first error
+
+SCRIPTPATH="$( cd "$(dirname "$0")" || exit >/dev/null 2>&1 ; pwd -P )"
+export PATH="$PATH:$SCRIPTPATH/../../tools/apigee-sackmesser/bin"
+
+PROJECT_ID=$(gcloud config get-value project)
+GCP_REGION=europe-west2
+
+# Generate the gRCP Gateway based on the proto file
+rm -rdf generated || true
+./generate-gateway.sh --proto-path ./examples/currency.proto
+
+# Build the gRPC Gateway
+(cd generated/gateway && CGO_ENABLED=0 go build -o grpcgateway .)
+
+# Build the grpc-gateway container and push it to Artifact Registry
+(cd generated/gateway && docker build -t grpc-gateway:latest .)
+
+DOCKER_REPO="devrel"
+REPO_LOCATION="europe"
+
+if [ -z "$(gcloud artifacts repositories describe $DOCKER_REPO \
+   --location=$REPO_LOCATION \
+   --project "$PROJECT_ID" \
+   --format='get(name)')" ]; then \
+
+  gcloud artifacts repositories create $DOCKER_REPO \
+      --repository-format=docker \
+      --location=$REPO_LOCATION \
+      --project="$PROJECT_ID"
+fi
+
+IMAGE_PATH="$REPO_LOCATION-docker.pkg.dev/$PROJECT_ID/$DOCKER_REPO/grpc-gateway"
+docker tag grpc-gateway:latest "$IMAGE_PATH:latest"
+docker push "$IMAGE_PATH"
+
+# Deploy grpc-gateway container to Cloud Run
+sed -i.bak "s|GRPC_GATEWAY_IMAGE|$IMAGE_PATH|g" "templates/cloud-run-service.yaml"
+
+gcloud run services replace templates/cloud-run-service.yaml \
+  --project "$PROJECT_ID" --region $GCP_REGION \
+  --platform managed
+
+# Generate and deploy an Apigee API proxy for the currency-service
+SA_EMAIL="apigee-test-cloudrun@$APIGEE_X_ORG.iam.gserviceaccount.com"
+
+if [ -z "$(gcloud iam service-accounts list --filter "$SA_EMAIL" --format="value(email)"  --project "$APIGEE_X_ORG")" ]; then
+    gcloud iam service-accounts create apigee-test-cloudrun \
+        --description="Apigee Test Cloud Run" --project "$APIGEE_X_ORG"
+fi
+
+gcloud run services add-iam-policy-binding currency-service \
+	 --member="serviceAccount:$SA_EMAIL" \
+	 --role='roles/run.invoker' \
+	 --region=$GCP_REGION \
+	 --platform=managed --project "$PROJECT_ID"
+
+CLOUD_RUN_URL=$(gcloud run services list --filter currency-service --format="value(status.url)" --limit 1)
+sed -i "s|CLOUD_RUN_URL|$CLOUD_RUN_URL|g" "templates/apiproxy/targets/default.xml"
+
+TOKEN="$(gcloud config config-helper --force-auth-refresh --format json | jq -r '.credential.access_token')"
+sackmesser deploy -d "$SCRIPTPATH/templates" -o "$APIGEE_X_ORG" -e "$APIGEE_X_ENV" -t "$TOKEN" --deployment-sa "$SA_EMAIL"
+
+# Test the Apigee API
+curl -X GET "https://$APIGEE_X_HOSTNAME/currency/v1/currencies"
+
+curl -X POST "https://$APIGEE_X_HOSTNAME/currency/v1/convert" \
+   -d '{"from": {"units": 3, "currency_code": "USD", "nanos": 0}, "to_code": "CHF"}'
+
+# Clean up
+  # undeploy and delete proxy
+  # delete CR service
+  # delete SA
+  # delete AR registry