S181 improve example (#38)

* improve the README

* reset-example, available-connectors tooling scripts

* .gitignore more appropriate to customer user

* tfsec workflow

.github/workflows/tfsec.yml

@@ -0,0 +1,21 @@
+name: 'tfsec'
+
+on:
+  push:
+    branches:
+      - '**'
+
+jobs:
+  tfsec:
+    name: tfsec
+    runs-on: ubuntu-latest
+
+    # q: what version of Terraform does this use??
+
+    steps:
+      - name: Clone repo
+        uses: actions/checkout@v3
+
+        # see: https://github.com/aquasecurity/tfsec-action
+      - name: tfsec
+        uses: aquasecurity/[email protected]

.gitignore

@@ -13,14 +13,14 @@ last-build.log
 # TODO: when this repo is ready to be used as template Terraform config, everything below should
 # be removed and these files committed to actual config repo IF they exist
 
-terraform.tfvars
+#terraform.tfvars
 
 # Terraform state - NOTE: for prod use, recommend you use a secure backend, such as S3/GCS for
 # terraform state - not storing state on local disk / repo
-terraform.tfstate**
-.terraform.tfstate.lock.info
+#terraform.tfstate**
+#.terraform.tfstate.lock.info
 
 # this *SHOULD* be committed to repo for actual configurations, but not to template repo.
-.terraform.lock.hcl
+#.terraform.lock.hcl
 
 

README.md

@@ -2,7 +2,7 @@
 
 [![Latest Release](https://img.shields.io/github/v/release/Worklytics/psoxy-example-gcp)](https://github.com/Worklytics/psoxy-example-gcp/releases/latest)
 ![build passing](https://img.shields.io/github/actions/workflow/status/Worklytics/psoxy-example-gcp/terraform_validate.yaml?label=build%20passing)
-
+![tfsec](https://img.shields.io/github/actions/workflow/status/Worklytics/psoxy-example-gcp/tfsec.yml?label=tfsec)
 
 This is a template repo for a Terraform configuration that deploys the [Worklytics pseudonymization
 proxy (psoxy)](https://github.com/Worklytics/psoxy) on GCP.
@@ -52,17 +52,89 @@ git clone https://github.com/{{YOUR_ORG_ID}}/{{YOUR_REPO_NAME}}.git
 ./init
 ```
 
-6. Review your `terraform.tfvars` file; customize as needed (eg, comment out data sources you don't
-   need, verify configuration options).
+
+6. Review your `terraform.tfvars` file and `main.tf`; customize as needed (eg, comment out
+   datasources you don't need).
+
+   In particular, if you're NOT using Google Workspace as a data source, remove (delete) the `.tf`
+   files named `google-*.tf` AND references to values from those files from the `main.tf` file.
+   (Our `./init` script *should* have removed these for you)
+
+   Similiarly, if you're NOT using Microsoft 365 as a data source, remove (delete) the `.tf`
+   files named `msft-365-*.tf` AND references to values from those files from the `main.tf` file.
+   (Our `./init` script *should* have removed these for you)
+
 
 7. Run `terraform plan` and review results to understand what will be created. Customize your
-   `terraform.tfvars` or `main.tf` file if needed.
+   `terraform.tfvars` or `main.tf` file if needed. (or push to your CI/CD system, if not running
+   locally)
 
 ```shell
 terraform plan
 ```
 
-8. Run `terraform apply` to create the resources.
+8. Run `terraform apply` to create the resources. (or push to your CI/CD system to do this
+   automatically)
 ```shell
 terraform apply
 ```
+
+9. The above steps have created or modified various files that you should commit a code repository
+   or otherwise preserve. In particular `terraform.tfvars`, `main.tf`, `terraform.tfstate` (if you
+   ran `terraform` locally) and `.terraform.lock.hcl` should be preserved. Please do `git add` for
+   each and then `git commit` to save your changes.
+
+## License
+
+The source code contained in this repo is licensed under the [Apache License, Version 2.0](LICENSE).
+
+Usage of terraform, psoxy, or other tooling invoked by scripts in this repo or described in the
+example tutorials it contains are each subject to their own license terms.
+
+## Support
+
+This example repo is maintained by [Worklytics](https://worklytics.co). Paid support is available.
+Please contact [[email protected]](mailto:[email protected]).
+
+6. Review your `terraform.tfvars` file and `main.tf`; customize as needed (eg, comment out
+   datasources you don't need).
+
+   In particular, if you're NOT using Google Workspace as a data source, remove (delete) the `.tf`
+   files named `google-*.tf` AND references to values from those files from the `main.tf` file.
+   (Our `./init` script *should* have removed these for you)
+
+   Similiarly, if you're NOT using Microsoft 365 as a data source, remove (delete) the `.tf`
+   files named `msft-365-*.tf` AND references to values from those files from the `main.tf` file.
+   (Our `./init` script *should* have removed these for you)
+
+
+7. Run `terraform plan` and review results to understand what will be created. Customize your
+   `terraform.tfvars` or `main.tf` file if needed. (or push to your CI/CD system, if not running
+   locally)
+
+```shell
+terraform plan
+```
+
+8. Run `terraform apply` to create the resources. (or push to your CI/CD system to do this
+   automatically)
+```shell
+terraform apply
+```
+
+9. The above steps have created or modified various files that you should commit a code repository
+   or otherwise preserve. In particular `terraform.tfvars`, `main.tf`, `terraform.tfstate` (if you
+   ran `terraform` locally) and `.terraform.lock.hcl` should be preserved. Please do `git add` for
+   each and then `git commit` to save your changes.
+
+## License
+
+The source code contained in this repo is licensed under the [Apache License, Version 2.0](LICENSE).
+
+Usage of terraform, psoxy, or other tooling invoked by scripts in this repo or described in the
+example tutorials it contains are each subject to their own license terms.
+
+## Support
+
+This example repo is maintained by [Worklytics](https://worklytics.co). Paid support is available.
+Please contact [[email protected]](mailto:[email protected]).

available-connectors

@@ -0,0 +1,39 @@
+#!/bin/bash
+
+RED='\e[0;31m'
+BLUE='\e[0;34m'
+NC='\e[0m' # No Color
+
+# default to .terraform/modules/psoxy/ if no argument provided (this is the correct thing when
+# running from the root of the psoxy repo)
+PSOXY_BASE_DIR=${1:-".terraform/modules/psoxy/"}
+
+MODULE_PATH="${PSOXY_BASE_DIR}infra/modules/worklytics-connector-specs"
+
+if [ ! -d "$MODULE_PATH" ]; then
+  printf "${RED}Connector specs module not found at ${MODULE_PATH}.${NC}\n"
+  printf "(if testing from main psoxy repo, run this script as ${BLUE}./tools/available-connects.sh ./${NC})\n"
+  printf "Exiting.${NC}\n"
+  exit 1
+fi
+
+
+# init worklytics-connector-specs module as if it's a terraform config, so subsequent 'console' call
+# will work
+terraform -chdir="${MODULE_PATH}" init >> /dev/null
+CLI_VARS="-var=include_msft=true -var=include_google_workspace=true"
+AVAILABLE_CONNECTORS=$(echo "jsonencode(tolist(keys(local.all_default_connectors)))" | terraform -chdir="${MODULE_PATH}" console $CLI_VARS)
+
+# clean up what the init did above
+rm -rf "${MODULE_PATH}/.terraform" 2> /dev/null
+rm "${MODULE_PATH}/.terraform.lock.hcl" 2> /dev/null
+
+if [ -z "$AVAILABLE_CONNECTORS" ]; then
+  printf "${RED}Failed to generate list of available connectors${NC} Contact support for assistance.\n"
+else
+  printf "The following connector configurations are available for the current version of the proxy Terraform modules you're using:\n"
+  echo "$AVAILABLE_CONNECTORS" | jq -r 'fromjson | .'
+
+  printf "To use a connector, add its id from the above list to ${BLUE}enabled_connectors${NC} in your ${BLUE}terraform.tfvars${NC} file.\n";
+  printf "Review the documentation for the connector at ${BLUE}https://docs.worklytics.co/psoxy/sources${NC} for more information.\n"
+fi

reset-example

@@ -0,0 +1,64 @@
+#!/bin/bash
+
+
+# colors
+RED='\e[0;31m'
+BLUE='\e[0;34m'
+NC='\e[0m' # No Color
+
+
+# warn user that will delete a bunch of files
+printf "This script will ${RED}delete${NC} the your local terraform state, variable files, etc, to "
+printf "reset to example template prior to ${BLUE}./init${NC} and any terraform init/plan/apply you've done.\n"
+printf "If you have ${RED}NOT${NC} committed these files and/or your local changes, they will be lost.\n"
+printf "Do you want to continue? (y/N): "
+read -r response
+if [[ ! "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
+    printf "Exiting...\n"
+    exit 0
+fi
+
+# resets example to state prior to `./init`
+rm .terraform.lock.hcl 2>/dev/null
+rm build 2>/dev/null
+rm update-bundle 2>/dev/null
+rm psoxy-* 2>/dev/null
+rm -rf .terraform 2>/dev/null
+rm terraform.tfvars 2>/dev/null
+rm terraform.tfstate 2>/dev/null
+
+# restore main.tf, if modified
+printf "Restoring ${BLUE}main.tf${NC} configuration file ...\n"
+git checkout HEAD -- main.tf
+
+# check source-specific files that may have been deleted
+FILES=("msft-365.tf" "msft-365-variables.tf" "google-workspace.tf" "google-workspace-variables.tf")
+
+check_and_restore_file() {
+    local file="$1"
+
+    # Check the git status to find out if the file was deleted
+    if git status --short | grep -q "^ D $file"; then
+        # The file is deleted, restore it from the HEAD
+        printf "Configuration file ${BLUE}$file${NC} was deleted, restoring...\n"
+        git checkout HEAD -- "$file"
+
+        if [ $? -eq 0 ]; then
+            printf "${BLUE}$file${NC} has been successfully restored.\n"
+        else
+            printf "${RED}Error occurred while restoring '$file'${NC}\n"
+            return 1
+        fi
+    fi
+}
+
+# Loop through the files and pass each one to the check_and_restore_file function
+for file in "${FILES[@]}"; do
+    check_and_restore_file "$file"
+done
+
+if [[ -f upgrade-terraform-modules ]]; then
+  rm upgrade-terraform-modules
+fi
+
+