diff --git a/docs/docs/explanations/advanced-provider-configuration.md b/docs/docs/explanations/advanced-provider-configuration.md index 2126e5c5..fd845d1b 100644 --- a/docs/docs/explanations/advanced-provider-configuration.md +++ b/docs/docs/explanations/advanced-provider-configuration.md @@ -98,6 +98,63 @@ amazon_web_services: permissions_boundary: arn:aws:iam::01234567890:policy/ ``` +### Launch Templates (Optional) + +Nebari supports configuring launch templates for your node groups, enabling you to customize settings like the AMI ID and pre-bootstrap commands. This is particularly useful if you need to use a custom AMI or perform specific actions before the node joins the cluster. + +:::warning +If you add a `launch_template` to an existing node group that was previously created without one, AWS will treat this as a change requiring the replacement of the entire node group. This action will trigger a reallocation of resources, effectively destroying the current node group and recreating it. This behavior is due to how AWS handles self-managed node groups versus those using launch templates with custom settings. +::: + +:::tip +To avoid unexpected downtime or data loss, consider creating a new node group with the launch template settings and migrating your workloads accordingly. This approach allows you to implement the new configuration without disrupting your existing resources. +::: + +#### Configuring a Launch Template + +To configure a launch template for a node group in your `nebari-config.yaml`, add the `launch_template` section under the desired node group: + +```yaml +amazon_web_services: + region: us-west-2 + kubernetes_version: "1.18" + node_groups: + custom-node-group: + instance: "m5.large" + min_nodes: 1 + max_nodes: 5 + gpu: false # Set to true if using GPU instances + launch_template: + # Replace with your custom AMI ID + ami_id: ami-0abcdef1234567890 + # Command to run before the node joins the cluster + pre_bootstrap_command: | + #!/bin/bash + # This script is executed before the node is bootstrapped + # You can use this script to install additional packages or configure the node + # For example, to install the `htop` package, you can run: + # sudo apt-get update + # sudo apt-get install -y htop" +``` + +**Parameters:** + +- `ami_id` (Optional): The ID of the custom AMI to use for the nodes in this group; this assumes the AMI provided is an EKS-optimized AMI derivative. If specified, the `ami_type` is automatically set to `CUSTOM`. +- `pre_bootstrap_command` (Optional): A command or script to execute on the node before + it joins the Kubernetes cluster. This can be used for custom setup or configuration + tasks. The format should be a single string in conformation with the shell syntax. + This command is injected in the `user_data` field of the launch template. For more + information, see [User Data](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html). + +> If you're using a `launch_template` with a custom `ami_id`, there's an issue with updating the `scaling.desired_size` via Nebari configuration (terraform). To scale up, you must recreate the node group or adjust the scaling settings directly in the AWS Console UI (recommended). We are aware of this inconsistency and plan to address it in a future update. + +:::note +If an `ami_id` is not provided, AWS will use the default Amazon Linux 2 AMI for the +specified instance type. You can find the latest optimized AMI IDs for Amazon EKS in your +cluster region by inspecting its respective SSM parameters. For more information, see +[Retrieve recommended Amazon Linux AMI IDs](https://docs.aws.amazon.com/eks/latest/userguide/retrieve-ami-id.html). +::: +