Per-Project Prefixes in Project Factory?

[robrankin]

TL;DR

We'd like to be able to customise the prefix in certain areas of FAST (Network and Project Factory at least) without adopting Tenant Factory.

There are no "tenants" in our organization, at least with any autonomy. A "tenant" in our world is a customer whom we deploy and manage a product for.

Problem:

We don't see any need or benefit at this point to adopting the FAST Tenant Factory approach. There's chance we're wrong about this, but it seems to us that the FAST definition of tenant doesn't really fit our organization.

What we'd like to do is just be able to change the prefix used for each customers set of resources from main to cst1 and cst2 (see the below example).

It appears that Project Factory doesn't allow this, as the prefix is enforced using data_overrides, unless I'm missing something?

I've worked up a set of changes to the Project Factory stage and the Project Factory module that allow a custom prefix per project, and I'm happy to open a PR for them, but wanted to see if there was any appetite for this, as the explicit enforcement of a single prefix in the PF stage seems quite intentional.

Example:

We have a number of customers, to whom we provide a set of products/services. Those services comes in the form of a set of environments arranged along the usual lines of dev/stage/prod, but also separated by product lines. We may have a customer who takes multiple products, resulting in something like the following, using the hierarchy structure in Project Factory:

├── cst1
│   ├── _config.yaml
│   └── prd
│       ├── _config.yaml
│       ├── env01
│       │   ├── _config.yaml
│       │   └── ce01.yaml
│       └── env02
│           ├── _config.yaml
│           └── ce01.yaml
├── cst2
│   ├── _config.yaml
│   └── prd
│       ├── _config.yaml
│       ├── env01
│       │   ├── _config.yaml
│       │   └── ce01.yaml
│       └── env02
│           ├── _config.yaml
│           └── ce01.yaml
└── main
    ├── _config.yaml
    └── dev
        ├── _config.yaml
        ├── env01
        │   ├── _config.yaml
        │   └── ce01.yaml
        └── env02
            ├── _config.yaml
            └── ce01.yaml

The "owning" company here is just "main" and thats the default prefix used across the organization and FAST.

cst1 and cst2 are the customers, and they've each taken a 2 "products", here called env01 and env02.

The "products" deployed to those environments are developed in the main owners dev type environments, but deployed to the customers prd type envs.

In each customers ce01.yaml project definition the only real change is adding a prefix key:

data/hierarchy/cst1/prd/env01/ce01.yaml

name: prd-env01-ce01
prefix: cst1

The resulting set of projects would be something like:

  ~ projects         = {
      + cst1-prd-env01-ce01 = {
          + number     = (known after apply)
          + project_id = "cst1-prd-env01-ce01"
        }
      + cst1-prd-env02-ce01 = {
          + number     = (known after apply)
          + project_id = "cst1-prd-env02-ce01"
        }
      + cst2-prd-env01-ce01 = {
          + number     = (known after apply)
          + project_id = "cst2-prd-env01-ce01"
        }
      + cst2-prd-env02-ce01 = {
          + number     = (known after apply)
          + project_id = "cst2-prd-env02-ce01"
        }
      + main-dev-env01-ce01 = {
          + number     = (known after apply)
          + project_id = "main-dev-env01-ce01"
        }
      + main-dev-env02-ce01 = {
          + number     = (known after apply)
          + project_id = "main-dev-env02-ce01"
        }
        # (1 unchanged attribute hidden)
    }

[ludoo]

Agreed, tenants are probably overkill. You should already be able to do what you need: just change the project factory stage so prefix is not enforced via overrides, but passed is as default (or not even that). At that point, you should be able to declare customized prefixes for each project.

[ludoo]

So in your case, projects created for tenants use a prefix that maps to the tenant (potentially "locking" a project id they might be interested in themselves in their org), and not to your org/prefix?

This seems slightly strange and I'm not sure it's a model I'd want to have directly supported in the module. :)

The projects key issue we can address ASAP while we keep discussing, feel free to send a PR or log an issue and I'll tackle it as soon as I have some spare time.

[robrankin]

So in your case, projects created for tenants use a prefix that maps to the tenant (potentially "locking" a project id they might be interested in themselves in their org), and not to your org/prefix?

I suppose thats one way of looking at it, but given the allowed prefix length (9?) plus the possible combinations across the rest of the project ID, it seems a little unlikely there would be a problem in reality.

Personally I prefer to have all parts of a resouce name have meaning, thats easy to convey and understand, and having the same prefix across all of them uses valuable naming space without conveying much. Given those resources exist within a GCP Org, the ultimate ownership and responsibility is already implied, so IMO having the same prefix everywhere doesn't add much. Using something like a prefix to identify different groups adds meaning.

All that said, I get your point, and being reluctant to add it to the codebase. I'd just leave one last arguement that moving the current system to "default plus customisable where the user requires it" adds flexibility without (I think?) a lot of complexity, at least from the POV of the users of this codebase.

The projects key issue we can address ASAP while we keep discussing, feel free to send a PR or log an issue and I'll tackle it as soon as I have some spare time.

I'll see bout PR'ng what I've done so far, give me a day or three.

[robrankin]

Opened a PR for just allowing duplicate filenames in the project hierarchy. Not sure about Windows though, will need to find a Windows machine to double check with.

[ludoo]

default plus customisable where the user requires it

absolutely, I'm just wary of adding complexity to cover individual corner cases given those can be implemented downstream with few changes to the module (like you did), let's take it one step at a time and see where we end up, if that's ok :)

checking the PR

[robrankin]

Tested on Windows, and fileset() returns forward slash / as the path separator so the change in the PR shouldnt cause any platform related issues.

[jackspyder]

It would also be good if, in the automation block you can set backend storage account location.

[ludoo]

It would also be good if, in the automation block you can set backend storage account location.

can you clarify what you mean with this?