Multitenancy usage - organization-level vs. tenant-level stages

[mnoon01]

Regarding tenants defined in a GCP organization-level’s resource management stage, i.e., lightweight tenants, I note from the documentation at https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/blob/master/fast/stages/1-resman/README.md that such tenants (apparently) allow:

subsequent Terraform stages to create network resources for each tenant which are centrally managed and connected to central networking, and tenants themselves to optionally manage their own networking and application projects.

Accordingly, I have three questions.

Firstly, as to the above quote, can you confirm that the former tenants referred to are setup in the organisation-level stages and the latter tenants referred to are (actually) setup in the tenant-level stages (which have separate resource hierarchies)?

Secondly, if so, what "network resources for each [lightweight] tenant" are envisaged being created?

Thirdly, certain lightweight tenant’s outputs don’t have corresponding input variable declarations in any subsequent stage. For example, billing_account and organization variables nested under core. Also, variables of the same names except they’re unnested, which are output with null values that are invalid in subsequent stages. So, how are those outputs intended to be used?

[ludoo]

I cannot unravel what you ask :)

Firstly, as to the above quote, can you confirm that the former tenants referred to are setup in the organisation-level stages and the latter tenants referred to are (actually) setup in the tenant-level stages (which have separate resource hierarchies)?

The simple explanation is our two tenant models differ in the level of support they offer for FAST stages:

• "hard" tenants are designed to emulate FAST stage 0 and 1 using the tenant folder as the root instead of the org, so that stages 2+ (networking, security, data platform, etc.) can be applied at the tenant level
• "soft" tenants do not aim for FAST compatibility, because the assumption is they will leverage central networking (stage 2 run at the org level), and/or custom code that implements specialized designs

Hard tenants are complex to setup, as you need to run tenant-level stage 0 and 1 for each tenant to set up the "org emulation" resources that give each tenant full FAST compatibility for stage 2+.

Soft tenants are easy to setup, as they are basically folder-rooted contexts with a scoped service account able to manage it, and some minimal organization policies like Domain Restricted Sharing if tenant uses a different Cloud Identity.

In our experience, you use hard tenancy where you have few tenants with a high degree of autonomy in the same org, to basically set up "mini orgs" for them. You use soft tenancy where you have many (ten or more) tenants and a "tenant template" that sets up each of them with connection to centralized networking, etc.

As for output and input variables not matching, yes that's expected and it's how FAST operates in general, where the producer stage exposes everything that is possibly of interest to whoever follows, and consumer stages only declare what they need in variables.

For lightweight tenants, a single tfvars file decouples them from FAST as they do not need to be aware of the org-level stages, so that file contains every piece of data they might need from the org-level stages.

[mnoon01]

Thanks @ludoo, I appreciate the explanation. However, your descriptions of the "hard" and "soft" tenants appear to be at odds. Specifically, where you say:

"hard" tenants are designed to emulate FAST stage 0 and 1 using the tenant folder as the root instead of the org, […]

I assume you meant to say that "hard" tenants use the org, and "soft" tenants use the tenant folder, as their respective root. In which case your "soft" tenants accord with what I referred to as "lightweight tenants" from the FAST documentation. Otherwise, your explanation is the confirmation sought in my first question.

Accordingly, regarding my second question, perhaps that documentation is ambiguous. However, adopting your more convenient nomenclature, I have set up hard tenants and soft tenants each of which I recognise can have separate billing accounts and or secondary organizations (which I understand you refer to as "mini orgs"). As previously mentioned, the documentation notes stage 2 of FAST can "create network resources for each [soft] tenant". So, pausing on that word "each", I remain interested as to what such network resources are envisaged being created?

Conversely, you note from your experience that you use 'a "tenant template" that sets up each [soft tenant] with [a] connection to centralized networking, etc.', which statement to me more accurately characterises the soft tenant model than the FAST documentation does. If not, please let us know.

Regarding my third question and your last two paragraphs, I appreciate that not all cross-stage variables will match. However, the unnested variables billing_account and organization that are output with null values are invalid in any subsequent stage, whether that be a FAST stage 2+ or a consumer stage. So, if those nulls are expected behaviour, I remain uncertain as to how those outputs are intended to be used? Alternatively, have I encountered unexpected behaviour?

[ludoo]

Thanks @ludoo, I appreciate the explanation. However, your descriptions of the "hard" and "soft" tenants appear to be at odds. Specifically, where you say:

"hard" tenants are designed to emulate FAST stage 0 and 1 using the tenant folder as the root instead of the org, […]

I assume you meant to say that "hard" tenants use the org, and "soft" tenants use the tenant folder, as their respective root. In which case your "soft" tenants accord with what I referred to as "lightweight tenants" from the FAST documentation. Otherwise, your explanation is the confirmation sought in my first question.

No, I meant exactly what I wrote, and since I developed the feature you can trust me. :)

No tenant has access to the org from their own code / stages.

Hard tenants have an environment that simulates the org from the FAST PoV, and are able to run stages 2 and 3 independently in it.

Soft tenants have a folder with elevated permissions, but they lack all the machinery that would enable them to run FAST stages directly inside it.

Accordingly, regarding my second question, perhaps that documentation is ambiguous. However, adopting your more convenient nomenclature, I have set up hard tenants and soft tenants each of which I recognise can have separate billing accounts and or secondary organizations (which I understand you refer to as "mini orgs"). As previously mentioned, the documentation notes stage 2 of FAST can "create network resources for each [soft] tenant". So, pausing on that word "each", I remain interested as to what such network resources are envisaged being created?

I refer to the tenant folder as a "mini org". What tenants can have are a billing account, and an external Cloud Identity/Workspace for users (which is different from a GCP org).

You're mixing up stuff, I suggest you to start from scratch and reread what I wrote. :) Soft tenants cannot run FAST stages. Hard tenants can. What networking resources can you then use in soft tenants? Custom designs and code. Soft tenants are mostly a way to decouple permissions from the org in a repeatable and standardized manner.

Conversely, you note from your experience that you use 'a "tenant template" that sets up each [soft tenant] with [a] connection to centralized networking, etc.', which statement to me more accurately characterises the soft tenant model than the FAST documentation does. If not, please let us know.

Yep, this is it. But that template is not something we provide, as I wrote above soft tenant are just a way of decoupling, to allow for custom code / designs at the tenant level.

Regarding my third question and your last two paragraphs, I appreciate that not all cross-stage variables will match. However, the unnested variables billing_account and organization that are output with null values are invalid in any subsequent stage, whether that be a FAST stage 2+ or a consumer stage. So, if those nulls are expected behaviour, I remain uncertain as to how those outputs are intended to be used? Alternatively, have I encountered unexpected behaviour?

They are valid, we run stages frequently in our own orgs or for customers, and nightly via a pipeline. If you are seeing errors please open an issues and be specific, and we will help you figure out what wrong in your configuration.