Skip to content

Commit

Permalink
AKS Baseline regulated - Bicep migration - Networking files (#63)
Browse files Browse the repository at this point in the history
Co-authored-by: Chad Kittel <[email protected]>
Co-authored-by: Fernando Antivero <[email protected]>
  • Loading branch information
3 people authored Sep 15, 2022
1 parent 73a7c0a commit 1991374
Show file tree
Hide file tree
Showing 19 changed files with 3,843 additions and 4,387 deletions.
5 changes: 2 additions & 3 deletions docs/deploy/04-subscription.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,9 +118,8 @@ It is recommended that your Azure _subscription_ have the **Azure Security Bench

1. Open the [**Regulatory Compliance** screen in Microsoft Defender for Cloud](https://portal.azure.com/#blade/Microsoft_Azure_Security/SecurityMenuBlade/22)
1. Click on **Manage compliance policies**
1. Click on your subscription
1. Ensure the **Azure Security Benchmark** is applied as the **Microsoft Defender for Cloud default policy**.
1. You'll want to ensure all relevant standards (e.g. **PCI DSS 3.2.1**) are **Enabled** under **Industry & regulatory standards**
1. Expand the tree, and click on your subscription
1. Navigate to **Security policy** blade since you'll want to ensure the **Azure Security Benchmark** and all relevant standards (e.g. **PCI DSS 3.2.1**) are **Enabled** under **Industry & regulatory standards**
1. The **Regulatory Compliance** dashboard in Microsoft Defender for Cloud might take an hour or two to reflect any modifications you've made.

**None of the above is required for this walkthrough**, but we want to ensure you're aware of these subscription-level policies and how you can enable them for your final implementation. All subscriptions containing PCI workloads should have the **PCI DSS 3.2.1** Industry & regulatory standards reports enabled, which _requires_ that the **Azure Security Benchmark** is applied as the default policy.
Expand Down
4 changes: 2 additions & 2 deletions docs/deploy/05-networking-hub.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Egressing your spoke traffic through a hub network (following the hub-spoke mode

After executing these steps you'll have the `rg-enterprise-networking-hubs` resource group populated with a regional virtual network (vnet), Azure Firewall, Azure Bastion, and Azure Monitor & Flow Log storage for network observability. No spokes will have been created yet, so the default firewall rules are maximally restrictive, as there is no expected outflow of traffic so none is allowed. We'll open up access on an as-needed bases throughout this walk through.

Specifically, you'll see networking/hub-region.v​_n_.json referenced a couple times in this process. Think of this as an evolution of a _single_ ARM template as the number and needs of the connected spokes change over time. You can diff the v​_n_ and v​_n+1_ ARM templates to see this progression over time. Typically your network team would have encapsulated this hub in a file named something like `hub-eastus2.json` and updated it in their source control as dependencies/requirements dictate. It likely would have not taken as many parameters as either, as those would be constants that could be easily defined directly in the template as the file would be specific to the region's spokes. To keep this reference implementation more flexible on what region you select, you'll be asked to provide deployment parameters and the filename can remain the generic name of hub-​_region_.
Specifically, you'll see networking/hub-region.v​_n_.bicep referenced a couple times in this process. Think of this as an evolution of a _single_ ARM template as the number and needs of the connected spokes change over time. You can diff the v​_n_ and v​_n+1_ ARM templates to see this progression over time. Typically your network team would have encapsulated this hub in a file named something like `hub-eastus2.bicep` and updated it in their source control as dependencies/requirements dictate. It likely would have not taken as many parameters as either, as those would be constants that could be easily defined directly in the template as the file would be specific to the region's spokes. To keep this reference implementation more flexible on what region you select, you'll be asked to provide deployment parameters and the filename can remain the generic name of hub-​_region_.

The examples that follow use `eastus2` as the primary region. You're welcome to change this in the ARM template parameters throughout this walkthrough. Clusters are regional resources; and the expectation is that your regional hub, regional spoke, and regional workload are all sharing the same region. So if you make a change to the region, be sure you change it in _all_ places along the way. For a reference architecture of a general-purpose, multi-region cluster, see [AKS Baseline for Multi-Region Topology](https://github.com/mspnp/aks-baseline-multi-region).

Expand All @@ -29,7 +29,7 @@ Since this walkthrough is expected to be deployed isolated from existing infrast

```bash
# [This takes about eight minutes to run.]
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v0.json -p location=eastus2
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v0.bicep -p location=eastus2
```

The hub deployment will output the following:
Expand Down
8 changes: 4 additions & 4 deletions docs/deploy/06-aks-jumpboximage.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,20 +40,20 @@ You are going to be using Azure Image Builder to generate a Kubernetes-specific
RESOURCEID_VNET_HUB=$(az deployment group show -g rg-enterprise-networking-hubs -n hub-region.v0 --query properties.outputs.hubVnetId.value -o tsv)

# [This takes about one minute to run.]
az deployment group create -g rg-enterprise-networking-spokes -f networking/spoke-BU0001A0005-00.json -p location=eastus2 hubVnetResourceId="${RESOURCEID_VNET_HUB}"
az deployment group create -g rg-enterprise-networking-spokes -f networking/spoke-BU0001A0005-00.bicep -p location=eastus2 hubVnetResourceId="${RESOURCEID_VNET_HUB}"
```

1. Update the regional hub deployment to account for the requirements of the spoke.

Now that the first spoke network is created, the hub network's firewall needs to be updated to support the Azure Image Builder process that will execute in there. The hub firewall does NOT have any default permissive egress rules, and as such, each needed egress endpoint needs to be specifically allowed. This deployment builds on the prior with the added allowances in the firewall.
> :eyes: If you're curious to see what changed in the regional hub, [view the diff](https://diffviewer.azureedge.net/?l=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v0.json&r=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v1.json).
> :eyes: If you're curious to see what changed in the regional hub, [view the diff](https://diffviewer.azureedge.net/?l=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v0.bicep&r=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v1.bicep).

```bash
RESOURCEID_SUBNET_AIB=$(az deployment group show -g rg-enterprise-networking-spokes -n spoke-BU0001A0005-00 --query properties.outputs.imageBuilderSubnetResourceId.value -o tsv)
# [This takes about five minutes to run.]
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v1.json -p location=eastus2 aksImageBuilderSubnetResourceId="${RESOURCEID_SUBNET_AIB}"
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v1.bicep -p location=eastus2 aksImageBuilderSubnetResourceId="${RESOURCEID_SUBNET_AIB}"
```

### Build and deploy the jump box image
Expand All @@ -62,7 +62,7 @@ Now that we have our image building network created, egressing through our hub,
1. Download the ARM templates from the AKS Jump Box Image Builder repository.
Ideally core templates like this would be part of your private Bicep registry. For this walk through, we are simply downloading the remote Bicep templates locally for execution.
Ideally core templates like this would be part of your private Bicep registry. For this walk through, we are simply downloading the remote ARM templates locally for execution.
```bash
wget -B https://raw.githubusercontent.com/mspnp/aks-jumpbox-imagebuilder/main/ -x -nH --cut-dirs=3 -i jumpbox/jumpbox-bicep.txt -P jumpbox
Expand Down
6 changes: 3 additions & 3 deletions docs/deploy/08-cluster-networking.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,22 +23,22 @@ Your `rg-enterprise-networking-spokes` will be populated with the dedicated regi
RESOURCEID_VNET_HUB=$(az deployment group show -g rg-enterprise-networking-hubs -n hub-region.v0 --query properties.outputs.hubVnetId.value -o tsv)

# [This takes about five minutes to run.]
az deployment group create -g rg-enterprise-networking-spokes -f networking/spoke-BU0001A0005-01.json -p location=eastus2 hubVnetResourceId="${RESOURCEID_VNET_HUB}"
az deployment group create -g rg-enterprise-networking-spokes -f networking/spoke-BU0001A0005-01.bicep -p location=eastus2 hubVnetResourceId="${RESOURCEID_VNET_HUB}"
```

1. Update the regional hub deployment to account for the runtime requirements of the virtual network.

This is an evolution of same hub template you used before, but now updated with Azure Firewall rules specific to this AKS cluster infrastructure.

> :eyes: If you're curious to see what changed in the regional hub, [view the diff](https://diffviewer.azureedge.net/?l=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v1.json&r=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v2.json).
> :eyes: If you're curious to see what changed in the regional hub, [view the diff](https://diffviewer.azureedge.net/?l=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v1.bicep&r=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v2.bicep).
```bash
RESOURCEID_SUBNET_AIB=$(az deployment group show -g rg-enterprise-networking-spokes -n spoke-BU0001A0005-00 --query properties.outputs.imageBuilderSubnetResourceId.value -o tsv)
RESOURCEID_SUBNET_NODEPOOLS="['$(az deployment group show -g rg-enterprise-networking-spokes -n spoke-BU0001A0005-01 --query "properties.outputs.nodepoolSubnetResourceIds.value | join ('\',\'',@)" -o tsv)']"
RESOURCEID_SUBNET_JUMPBOX=$(az deployment group show -g rg-enterprise-networking-spokes -n spoke-BU0001A0005-01 --query properties.outputs.jumpboxSubnetResourceId.value -o tsv)
# [This takes about seven minutes to run.]
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v2.json -p location=eastus2 aksImageBuilderSubnetResourceId="${RESOURCEID_SUBNET_AIB}" nodepoolSubnetResourceIds="${RESOURCEID_SUBNET_NODEPOOLS}" aksJumpboxSubnetResourceId="${RESOURCEID_SUBNET_JUMPBOX}"
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v2.bicep -p location=eastus2 aksImageBuilderSubnetResourceId="${RESOURCEID_SUBNET_AIB}" nodepoolSubnetResourceIds="${RESOURCEID_SUBNET_NODEPOOLS}" aksJumpboxSubnetResourceId="${RESOURCEID_SUBNET_JUMPBOX}"
```
### Next step
Expand Down
4 changes: 2 additions & 2 deletions modules/subscriptionPolicyAssignment.bicep
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ param enforcementMode string = 'Default'
@minLength(4)
param location string

@description('The name of the policy set to assign.')
@description('The name of the policy or policy set to assign.')
@minLength(36)
@maxLength(36)
param policyDefinitionSetName string
Expand All @@ -33,7 +33,7 @@ var builtIntPolicyDefinitionSetId = subscriptionResourceId('Microsoft.Authorizat

@description('Assignment of policy')
resource policyAssignment 'Microsoft.Authorization/policyAssignments@2021-06-01' = {
name: guid(builtIntPolicyDefinitionSetId)
name: guid(builtIntPolicyDefinitionSetId, subscription().id)
identity: {
type: 'SystemAssigned'
}
Expand Down
10 changes: 5 additions & 5 deletions networking/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ This is part of the Azure Kubernetes Service (AKS) Baseline Cluster for Regulate

## Files

* [`hub-region.v0.json`](./hub-region.v0.json) is a file that defines a generic regional hub. All regional hubs can generally be considered a fork of this base template.
* [`hub-region.v1.json`](./hub-region.v1.json) is an updated version that defines a specific region's hub (for example, it might be named `hub-eastus2.json`). This is the long-lived template that defines this specific region's hub. This version has support for our image builder process.
* [`hub-region.v2.json`](./hub-region.v2.json) is an even more updated version that defines a specific region's hub (it would still be the same named `hub-eastus2.json` file). This version has support for our image builder process plus our AKS cluster's needs.
* [`spoke-BU0001A0005-00.json`](./spoke-BU0001A0005-00.json) is a file that defines a specific spoke in the topology. A spoke is created for each workload in a business unit, hence the naming pattern in the file name. This spoke contains the networking resources for the image builder process.
* [`spoke-BU0001A0005-01.json`](./spoke-BU0001A0005-01.json) is a file that defines a specific spoke in the topology. This spoke contains the networking resources for the AKS cluster.
* [`hub-region.v0.bicep`](./hub-region.v0.bicep) is a file that defines a generic regional hub. All regional hubs can generally be considered a fork of this base template.
* [`hub-region.v1.bicep`](./hub-region.v1.bicep) is an updated version that defines a specific region's hub (for example, it might be named `hub-eastus2.bicep`). This is the long-lived template that defines this specific region's hub. This version has support for our image builder process.
* [`hub-region.v2.bicep`](./hub-region.v2.bicep) is an even more updated version that defines a specific region's hub (it would still be the same named `hub-eastus2.bicep` file). This version has support for our image builder process plus our AKS cluster's needs.
* [`spoke-BU0001A0005-00.bicep`](./spoke-BU0001A0005-00.bicep) is a file that defines a specific spoke in the topology. A spoke is created for each workload in a business unit, hence the naming pattern in the file name. This spoke contains the networking resources for the image builder process.
* [`spoke-BU0001A0005-01.bicep`](./spoke-BU0001A0005-01.bicep) is a file that defines a specific spoke in the topology. This spoke contains the networking resources for the AKS cluster.

Your organization will likely have its own standards for their hub-spoke or vwan implementation. Be sure to follow your organizational guidelines.

Expand Down
Loading

0 comments on commit 1991374

Please sign in to comment.