From 50a4b6f0e7af3d24cfb7f32ccbc10c3b15747588 Mon Sep 17 00:00:00 2001 From: Connor Catlett Date: Fri, 16 Aug 2024 14:54:48 +0000 Subject: [PATCH] Document metadata requirement and available sources Signed-off-by: Connor Catlett --- docs/install.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/install.md b/docs/install.md index 682bd8e865..ec4502864c 100644 --- a/docs/install.md +++ b/docs/install.md @@ -8,6 +8,31 @@ * Important: If you intend to use the Volume Snapshot feature, the [Kubernetes Volume Snapshot CRDs](https://github.com/kubernetes-csi/external-snapshotter/tree/master/client/config/crd) must be installed **before** the EBS CSI driver. For installation instructions, see [CSI Snapshotter Usage](https://github.com/kubernetes-csi/external-snapshotter#usage). +### Metadata + +The EBS CSI Driver uses a metadata source in order to gather necessary information about the environment to function. The driver currently supports two metadata sources: [IMDS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html) or Kubernetes. + +The controller `Deployment` can skip metadata if the region is provided via the `AWS_REGION` environment variable (Helm parameter `controller.region`). The node `DaemonSet` requires metadata and will not function without access to one of the sources. + +#### IMDS (EC2) Metadata + +If the driver is able to access IMDS, it will utilize that as a preferred source of metadata. The EBS CSI Driver supports IMDSv1 and IMDSv2 (and will prefer IMDSv2 if both are available). However, by default, [IMDSv2 uses a hop limit of 1](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html#instance-metadata-v2-how-it-works). That will prevent the driver from accessing IMDSv2 if run inside a container with the default IMDSv2 configuration. + +In order for the driver to access IMDS, it either must be run in host networking mode, or with a [hop limit of at least 2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-IMDS-existing-instances.html#modify-PUT-response-hop-limit). + +#### Kubernetes Metadata + +If the driver is unable to reach IMDS, it will fallback to using the Kubernetes API. For this metadata source to work, the driver pods must have access to the Kubernetes API server. Additionally, the Kubernetes node objects must include the following information: + +- Instance ID (in the `Node`'s `ProviderID`) +- Instance Type (in the label `node.kubernetes.io/instance-type`) +- Instance Region (in the label `topology.kubernetes.io/region`) +- Instance AZ (in the label `topology.kubernetes.io/zone`) + +These values are typically set by the [AWS CCM](https://github.com/kubernetes/cloud-provider-aws). You must have the AWS CCM or a similar tool installed in your cluster providing these values for Kubernetes metadata to function. + +Kubernetes metadata does not provide information about the number of ENIs or EBS volumes attached to an instance. Thus, when performing volume limit calculations, node pods using Kubernetes metadata will assume one ENI and one EBS volume (the root volume) is attached. + ## Installation ### Set up driver permissions The driver requires IAM permissions to talk to Amazon EBS to manage the volume on user's behalf. [The example policy here](./example-iam-policy.json) defines these permissions. AWS maintains a managed policy, available at ARN `arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy`.