v1.49.0

what

• Update depends_on and atmos describe dependents
• Restore checking for the context in imports with Go templates removed in #464
• Update README

why

• The settings.depends_on was incorrectly processed for cross-tenant, cross-environment and cross-staging dependencies
• When the context was not provided in imports with Go templates, we did not process Go templates. A change was introduced in #464 which does not work for all use-cases. Restoring the old functionality

v1.48.0

what

• Add overrides section to Atmos stack manifests
• Update docs https://atmos.tools/core-concepts/components/overrides/

why

Atmos supports the overrides pattern to modify component(s) configuration and behavior using the overrides section in Atmos stack manifests.

You can override the following sections in the component(s) configuration:

• vars
• settings
• env
• command

The overrides section can be used at the global level or at the Terraform and Helmfile levels.

Overrides Schema

The overrides section schema at the global level is as follows:

overrides:
  # Override the ENV variables for the components in the current stack manifest and all its imports
  env: {}
  # Override the settings for the components in the current stack manifest and all its imports
  settings: {}
  # Override the variables for the components in the current stack manifest and all its imports
  vars: {}
  # Override the command to execute for the components in the current stack manifest and all its imports
  command: "<command to execute>"

The overrides section schemas at the Terraform and Helmfile levels are as follows:

terraform:
  overrides:
    env: {}
    settings: {}
    vars: {}
    command: "<command to execute>"

helmfile:
  overrides:
    env: {}
    settings: {}
    vars: {}
    command: "<command to execute>"

You can include the overrides, terraform.overrides and helmfile.overrides sections in any Atmos stack manifest at any level of inheritance to override the configuration of all the Atmos components defined inline in the manifest and all its imports.

Use-case: Overrides for Teams

The overrides pattern is used to override the components only in a particular Atmos stack manifest and all the imported
manifests. This is different from the other configuration sections (e.g. vars, settings, env). If we define a vars, settings or env section at the global, Terraform or Helmfile levels, all the components in the top-level stack will get the updated configurations. On the other hand, if we define an overrides section in a stack manifest, only the components directly defined in the manifest and its imports will get the overridden values, not all the components in the top-level Atmos stack.

This is especially useful when you have Atmos stack manifests split per Teams, each Team manages a set of components, and you need to define a common configuration (or override the existing one) for the components that only a particular Team manages.

For example, we have two Teams: devops and testing.

The devops Team manifest is defined in stacks/teams/devops.yaml:

import:
  # The `devops` Team manages all the components defined in the following stack manifests:
  - catalog/terraform/top-level-component1

The testing Team manifest is defined in stacks/teams/testing.yaml:

import:
  # The `testing` Team manages all the components defined in the following stack manifests:
  - catalog/terraform/test-component
  - catalog/terraform/test-component-override

We can import the two manifests into a top-level stack manifest, e.g. tenant1/dev/us-west-2.yaml:

import:
  - mixins/region/us-west-2
  - orgs/cp/tenant1/dev/_defaults
  # Import all components that the `devops` Team manages
  - teams/devops
  # Import all components managed by the `testing` Team
  - teams/testing

Suppose that we want to change some variables in the vars and env sections and some config in the settings section for all the components that the testing Team manges, but we don't want to affect any components that the devops Team manages.

If we added a global or Terraform level vars, env or settings sections to the top-level manifest stacks/orgs/cp/tenant1/dev/us-west-2.yaml or to the Team manifest stacks/teams/testing.yaml, then all the components in the tenant1/dev/us-west-2 top-level stack would be modified, including those managed by the devops Team.

To solve this, we could individually modify the vars, env and settings sections in all the components managed by the testing Team, but the entire configuration would not be DRY and reusable. That's where the overrides pattern comes into play. To make the configuration DRY and configured only in one place, use the overrides section.

For example, we want to override some values in the env, vars and setings sections for all the components managed by the testing Team:

import:
  # The `testing` Team manages all the components defined in the following stack manifests:
  - catalog/terraform/test-component
  - catalog/terraform/test-component-override

# Global overrides.
# Override the variables, env, command and settings ONLY in the components managed by the `testing` Team.
overrides:
  env:
    # This ENV variable will be added or overridden in all the components managed by the `testing` Team
    TEST_ENV_VAR1: "test-env-var1-overridden"
  settings: {}
  vars: {}

# Terraform overrides.
# Override the variables, env, command and settings ONLY in the Terraform components managed by the `testing` Team.
# The Terraform `overrides` are deep-merged with the global `overrides`
# and takes higher priority (it will override the same keys from the global `overrides`).
terraform:
  overrides:
    settings:
      spacelift:
        # All the components managed by the `testing` Team will have the Spacelift stacks auto-applied
        # if the planning phase was successful and there are no plan policy warnings
        # https://docs.spacelift.io/concepts/stack/stack-settings#autodeploy
        autodeploy: true
    vars:
      # This variable will be added or overridden in all the Terraform components managed by the `testing` Team
      test_1: 1
    # The `testing` Team uses `tofu` instead of `terraform`
    # https://opentofu.org
    # The commands `atmos terraform <sub-command> ...` will execute the `tofu` binary
    command: tofu

# Helmfile overrides.
# Override the variables, env, command and settings ONLY in the Helmfile components managed by the `testing` Team.
# The Helmfile `overrides` are deep-merged with the global `overrides`
# and takes higher priority (it will override the same keys from the global `overrides`).
helmfile:
  overrides:
    env:
      # This ENV variable will be added or overridden in all the Helmfile components managed by the `testing` Team
      TEST_ENV_VAR2: "test-env-var2-overridden"

In the manifest above, we configure the following:

• The global overrides section to override the TEST_ENV_VAR1 ENV variable in the env section. All the Terraform and Helmfile components managed by the testing Team will get the ENV vars updated to test-env-var1-overridden.

• The Terraform-level terraform.overrides section to override some Spacelift configuration in the settings section, a variable in the vars section, and the tofu command to execute instead of terraform in the command section. All the Terraform components managed by the testing Team will be affected by the new values (but not the Helmfile components). The Terraform overrides are deep-merged with the global overrides and takes higher priority (it will override the same keys from the global overrides).

• The Helmfile-level helmfile.overrides section to override an ENV variable in the env section. All the Helmfile components managed by the testing Team will get the new ENV variable value (but not the Terraform components). The Helmfile overrides are deep-merged with the global overrides and takes higher priority (it will override the same keys from the global overrides).

To confirm that the components managed by the testing Team get the new values from the overrides sections, execute the following
commands:

atmos atmos describe component test/test-component -s tenant1-uw2-dev
atmos atmos describe component test/test-component-override -s tenant1-uw2-dev

You should see the following output:

# Final deep-merged `overrides` from all the global `overrides` and Terraform `overrides` sections
overrides:
  command: tofu
  env:
    TEST_ENV_VAR1: test-env-var1-overridden
  settings:
    spacelift:
      autodeploy: true
  vars:
    test_1: 1

# The `command` was overridden with the value from `terraform.overrides.command`
command: tofu

env:
  # The `TEST_ENV_VAR1` ENV variable was overridden with the value from `overrides.env.TEST_ENV_VAR1`
  TEST_ENV_VAR1: test-env-var1-overridden
  TEST_ENV_VAR2: val2

settings:
  spacelift:
    # The `autodeploy` setting was overridden with the value 
    # from `terraform.overrides.settings.spacelift.autodeploy`
    autodeploy: true
    workspace_enabled: true

vars:
  environment: uw2
  namespace: cp
  region: us-west-2
  stage: dev
  tenant: tenant1
  # The `test_1` variable was overridden with the value from `terraform.overrides.vars.test_1`
  test_1: 1

To confirm that the components managed by the devops Team are not affected by the overrides for the testing Team, execute the following command:

atmos atmos describe component top-level-component1 -s tenant1-uw2-dev

You should see the following output:

# The `command` is not overridden
command: terraform

# The component does not get the `overrides` section since it's not defined 
# for the components managed by the `devops` Team
overrides: {}

vars:
  <variables for 'top-level-component1' in the stack 'tenant1-uw2-dev'>

env:
  <ENV variables for 'top-level-component1' in the stack 'tenan...

v1.47.0

what

• Add vendor.yaml as a new way of vendoring components, stacks and other artifacts
• Add/update docs
  • https://atmos.tools/core-concepts/vendoring/
  • https://atmos.tools/cli/commands/vendor/pull/

why

Atmos natively supports the concept of "vendoring", which is making copies of the 3rd party components, stacks, and other artifacts in your own repo.

The vendoring configuration is described in the vendor.yaml manifest, which should be placed in the directory from which the atmos vendor pull command is executed, usually in the root of the infrastructure repo.

After defining the vendor.yaml manifest, all the remote artifacts can be downloaded by running the following command:

atmos vendor pull

To vendor a particular component or other artifact, execute the following command:

atmos vendor pull -c <component>

Vendoring Manifest

To vendor remote artifacts, create a vendor.yaml file similar to the example below:

apiVersion: atmos/v1
kind: AtmosVendorConfig
metadata:
  name: example-vendor-config
  description: Atmos vendoring manifest
spec:
  # `imports` or `sources` (or both) must be defined in a vendoring manifest
  imports:
    - "vendor/vendor2.yaml"
    # The imported file extension is optional. 
    # If an import is defined without an extension, the `.yaml` extension is assumed and used by default.
    - "vendor/vendor3"

  sources:
    # `source` supports the following protocols: OCI (https://opencontainers.org), Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP,
    # and all URL and archive formats as described in https://github.com/hashicorp/go-getter.
    # In 'source', Golang templates are supported  https://pkg.go.dev/text/template.
    # If 'version' is provided, '{{.Version}}' will be replaced with the 'version' value before pulling the files from 'source'.
    # Download the component from the AWS public ECR registry (https://docs.aws.amazon.com/AmazonECR/latest/public/public-registries.html).
    - component: "vpc"
      source: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:{{.Version}}"
      version: "latest"
      targets:
        - "components/terraform/infra/vpc3"
      # Only include the files that match the 'included_paths' patterns.
      # If 'included_paths' is not specified, all files will be matched except those that match the patterns from 'excluded_paths'.
      # 'included_paths' support POSIX-style Globs for file names/paths (double-star `**` is supported).
      # https://en.wikipedia.org/wiki/Glob_(programming)
      # https://github.com/bmatcuk/doublestar#patterns
      included_paths:
        - "**/*.tf"
        - "**/*.tfvars"
        - "**/*.md"
    - component: "vpc-flow-logs-bucket"
      source: "github.com/cloudposse/terraform-aws-components.git//modules/vpc-flow-logs-bucket?ref={{.Version}}"
      version: "1.323.0"
      targets:
        - "components/terraform/infra/vpc-flow-logs-bucket/{{.Version}}"
      excluded_paths:
        - "**/*.yaml"
        - "**/*.yml"

• The vendor.yaml vendoring manifest supports Kubernetes-style YAML config to describe vendoring configuration for components, stacks, and other artifacts. The file is placed into the directory from which the atmos vendor pull command is executed (usually the root of the repo)

• The sources in vendor.yaml support all protocols (local files, Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP), and all URL and archive formats as described in go-getter, and also the oci:// scheme to download artifacts from OCI registries.

• The targets in the sources support absolute and relative paths (relative to the directory where the command is executed)

• included_paths and excluded_paths support POSIX-style greedy Globs for filenames/paths (double-star/globstar ** is supported as well)

• The component attribute in each source is optional. It's used in the atmos vendor pull -- component <component command if the component is passed in. In this case, Atmos will vendor only the specified component instead of vendoring all the artifacts configured in the vendor.yaml manifest

• The source and targets attributes support Go templates
  and Sprig Functions. This can be used to templatise the source and targets paths with the artifact versions specified in the version attribute

• The imports section defines the additional vendoring manifests that are merged into the main manifest. Hierarchical imports are supported at many levels (one vendoring manifest can import another, which in turn can import other manifests, etc.). Atmos processes all imports and all sources in the imported manifests in the order they are defined

Hierarchical Imports in Vendoring Manifests

Use imports to split the main vendor.yaml manifest into smaller files for maintainability, or by their roles in the infrastructure.

For example, import separate manifests for networking, security, data management, CI/CD, and other layers:

imports:
  - "layers/networking"
  - "layers/security"
  - "layers/data"
  - "layers/analytics"
  - "layers/firewalls"
  - "layers/cicd"

Hierarchical imports are supported at many levels. For example, consider the following vendoring configurations:

# vendor.yaml
apiVersion: atmos/v1
kind: AtmosVendorConfig
metadata:
  name: example-vendor-config
  description: Atmos vendoring manifest
spec:
  imports:
    - "vendor/vendor2"
    - "vendor/vendor3"

  sources:
    - component: "vpc"
      source: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:{{.Version}}"
      version: "latest"
      targets:
        - "components/terraform/infra/vpc3"
    - component: "vpc-flow-logs-bucket"
      source: "github.com/cloudposse/terraform-aws-components.git//modules/vpc-flow-logs-bucket?ref={{.Version}}"
      version: "1.323.0"
      targets:
        - "components/terraform/infra/vpc-flow-logs-bucket/{{.Version}}"

# vendor/vendor2.yaml
apiVersion: atmos/v1
kind: AtmosVendorConfig
metadata:
  name: example-vendor-config-2
  description: Atmos vendoring manifest
spec:
  imports:
    - "vendor/vendor4"

  sources:
    - component: "my-vpc1"
      source: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:{{.Version}}"
      version: "1.0.2"
      targets:
        - "components/terraform/infra/my-vpc1"

# vendor/vendor4.yaml
apiVersion: atmos/v1
kind: AtmosVendorConfig
metadata:
  name: example-vendor-config-4
  description: Atmos vendoring manifest
spec:
  imports:
    - "vendor/vendor5"

  sources:
    - component: "my-vpc4"
      source: "github.com/cloudposse/terraform-aws-components.git//modules/vpc?ref={{.Version}}"
      version: "1.319.0"
      targets:
        - "components/terraform/infra/my-vpc4"

When you execute the atmos vendor pull command, Atmos processes the import chain and the sources in the imported manifests in the order they are defined:

• The vendor/vendor2.yaml and vendor/vendor3.yaml manifests (defined in the main vendor.yaml file) are imported
• The vendor/vendor2.yaml file is processed, and the vendor/vendor4.yaml manifest is imported
• The vendor/vendor4.yaml file is processed, and the vendor/vendor5.yaml manifest is imported
• Etc.
• Then all the sources from all the imported manifests are processed and the artifacts are downloaded into the paths defined by the targets

> atmos vendor pull

Processing vendor config file 'vendor.yaml'
Pulling sources for the component 'my-vpc6' from 'github.com/cloudposse/terraform-aws-components.git//modules/vpc?ref=1.315.0' into 'components/terraform/infra/my-vpc6'
Pulling sources for the component 'my-vpc5' from 'github.com/cloudposse/terraform-aws-components.git//modules/vpc?ref=1.317.0' into 'components/terraform/infra/my-vpc5'
Pulling sources for the component 'my-vpc4' from 'github.com/cloudposse/terraform-aws-components.git//modules/vpc?ref=1.319.0' into 'components/terraform/infra/my-vpc4'
Pulling sources for the component 'my-vpc1' from 'public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:1.0.2' into 'components/terraform/infra/my-vpc1'
Pulling sources for the component 'my-vpc2' from 'github.com/cloudposse/terraform-aws-components.git//modules/vpc?ref=1.320.0' into 'components/terraform/infra/my-vpc2'
Pulling sources for the component 'vpc' from 'public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:latest' into 'components/terraform/infra/vpc3'
Pulling sources for the component 'vpc-flow-logs-bucket' from 'github.com/cloudposse/terraform-aws-components.git//modules/vpc-flow-logs-bucket?ref=1.323.0' into 'components/terraform/infra/vpc-flow-logs-bucket/1.323.0'

Vendoring from OCI Registries

Atmos supports vendoring from OCI registries.

To specify a repository in an OCI registry, use the oci://<registry>/<repository>:tag scheme.

Artifacts from OCI repositories are downloaded as Docker image tarballs, then all the layers are processed, un-tarred and un-compressed, and the files are written into the directories specified by the targets attribute of each source.

For example, to vendor the vpc component from the public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc
AWS public ECR registry, use the following source:

source: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/v...

v1.46.0

what

• Update to Go 1.21
• Update/improve Atmos help for atmos-specific commands
• Improve atmos vendor pull command error handling
• Add Atmos custom commands to atmos.yaml and docs
• Add examples of Rego policies for Atmos components and stacks validation. Update docs. See https://atmos.tools/core-concepts/components/validation/
• Add Sprig functions support to Go templates in Atmos vendoring
• Improve context functionality in imports with Go templates. Update docs. See https://atmos.tools/core-concepts/stacks/imports/
• Add OCI support to atmos vendor command

why

• Update to Go 1.21.
  1.21 is the latest Go version with new features and improvements

• Update/improve Atmos help for atmos-specific commands.
  Atmos supports Terraform commands that are not Terraform native. Update the commands help to better describe the commands and don't call the native Terraform help on them (since it fails). For example:

atmos terraform clean --help

'atmos terraform clean' command deletes the following folders and files from the component's directory:

 - '.terraform' folder
 - folder that the 'TF_DATA_DIR' ENV var points to
 - '.terraform.lock.hcl' file
 - generated varfile for the component in the stack
 - generated planfile for the component in the stack
 - generated 'backend.tf.json' file

Usage: atmos terraform clean <component> -s <stack> <flags>

Use '--skip-lock-file' flag to skip deleting the lock file.

For more details refer to https://atmos.tools/cli/commands/terraform/clean

atmos terraform shell --help

'atmos terraform shell' command starts a new SHELL configured with the environment for an Atmos component in a Stack to allow executing all native terraform commands
inside the shell without using the atmos-specific arguments and flags.

Usage: atmos terraform shell <component> -s <stack>

The command does the following:

 - Processes the stack config files, generates the required variables for the Atmos component in the stack, and writes them to a file in the component's folder
 - Generates a backend config file for the Atmos component in the stack and writes it to a file in the component's folder (or as specified by the Atmos configuration setting)
 - Creates a Terraform workspace for the component in the stack
 - Drops the user into a separate shell (process) with all the required paths and ENV vars set
 - Inside the shell, the user can execute all Terraform commands using the native syntax

For more details refer to https://atmos.tools/cli/commands/terraform/shell

• Improve atmos vendor pull command error handling.
  If a user specifies the flags incorrectly, the command now shows a more descriptive error message. For example, executing atmos vendor pull infra/vpc2 will generate the following error message:

• Add Atmos custom commands to atmos.yaml and docs
  The new custom commands extend Atmos functionality. Especially useful is the atmos describe eks upgrade <eks_component> -s <stack> command that shows all the steps that needs to be performed to upgrade an EKS cluster to the next Kubernetes version, including checking for failing Pods and containers, EKS add-ons versions, and for the deleted k8s API versions (which need to be fixed by updating the Helm Charts before upgrading to the next Kubernetes version)

For example, to describe the steps to upgrade the eks Atmos component in the stack tenant1-ue1-dev, execute the following command:

atmos describe eks upgrade eks -s tenant1-ue1-dev

• Add examples of Rego policies for Atmos components and stacks validation. Update docs.
  Some useful examples of OPA Rego policies to validate Atmos components and stacks added to the examples and described in the docs

• Add Sprig functions support to Go templates in Atmos vendoring.
  The Sprig library provides over 70 template functions for Go’s template language.
  Atmos supported the Sprig functions in imports. This update adds it to Atmos vendoring and the functions can be used in the attributes in the component.yaml vendoring config file

• Improve context functionality in imports with Go templates.
  Before this update, imports with Go templates could use only one schema at the time in one file, either a list of strings (paths to the imported stack manifests), or a list of objects with context. Now, both schemas can be mixed in one import. For example:

import:
  - path: mixins/region/region_tmpl
    context:
      region: "{{ .region }}"
      environment: "{{ .environment }}"

  - path: "orgs/cp/{{ .tenant }}/{{ .stage }}/_defaults"
  - "catalog/eks/cluster/defaults"
  - "catalog/eks/karpenter/defaults"

Also, imports with Go templates now support two new attributes:

• skip_templates_processing - (boolean) Skip template processing for the imported file. Can be used if the imported file uses Go templates to configure external systems, e.g. Datadog

• ignore_missing_template_values - (boolean) Ignore the missing template values in the imported file. Can be used if the imported file uses Go templates to configure external systems, e.g. Datadog. In this case, Atmos will process all template values that are provided in the context, and will skip the missing values in the templates for the external systems without throwing an error. The ignore_missing_template_values setting is different from skip_templates_processing in that skip_templates_processing skips the template processing completely in the imported file, while ignore_missing_template_values processes the templates using the values provided in the context and skips all the missing values

For example:

import:
    - path: "<path_to_imported_file>"
      context: {}
      skip_templates_processing: true
      ignore_missing_template_values: false
    - path: "<path_to_imported_file>"
      context: {}
      skip_templates_processing: false
      ignore_missing_template_values: true

• Add OCI support to atmos vendor command

Atmos now supports vendoring components from OCI registries.

To specify a repository in an OCI registry, use the oci://<registry>/<repository>:tag scheme in the sources and mixins.

Components from OCI repositories are downloaded as Docker image tarballs, then all the layers are processed, un-tarred and un-compressed, and the component's source files are written into the component's directory.

For example, to vendor the vpc component from the public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc
AWS public ECR registry, use the following uri:

uri: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:latest"

The schema of a component.yaml file is as follows:

# This is an example of how to download a Terraform component from an OCI registry (https://opencontainers.org), e.g. AWS Public ECR

# 'component.yaml' in the component folder is processed by the 'atmos' commands:
# 'atmos vendor pull -c infra/vpc2' or 'atmos vendor pull --component infra/vpc2'

apiVersion: atmos/v1
kind: ComponentVendorConfig
metadata:
  name: stable/aws/vpc
  description: Config for vendoring of the 'stable/aws/vpc' component
spec:
  source:
    # Source 'uri' supports the following protocols: OCI (https://opencontainers.org), Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP,
    # and all URL and archive formats as described in https://github.com/hashicorp/go-getter
    # In 'uri', Golang templates are supported  https://pkg.go.dev/text/template
    # If 'version' is provided, '{{.Version}}' will be replaced with the 'version' value before pulling the files from 'uri'
    # Download the component from the AWS public ECR registry (https://docs.aws.amazon.com/AmazonECR/latest/public/public-registries.html)
    uri: "oci://public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:{{.Version}}"
    version: "latest"
    # Only include the files that match the 'included_paths' patterns
    # If 'included_paths' is not specified, all files will be matched except those that match the patterns from 'excluded_paths'
    # 'included_paths' support POSIX-style Globs for file names/paths (double-star `**` is supported)
    # https://en.wikipedia.org/wiki/Glob_(programming)
    # https://github.com/bmatcuk/doublestar#patterns
    included_paths:
      - "**/*.*"

When the following command is executed

atmos vendor pull -c infra/vpc2

Atmos downloads the Docker image tarball from the AWS public ECR repository public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:latest, then all the layers are processed, un-tarred and un-compressed, and the component's source files are written into the component's directory.

Pulling sources for the component 'infra/vpc2' 
from 'public.ecr.aws/cloudposse/components/terraform/stable/aws/vpc:latest' 
into 'examples/complete/components/terraform/infra/vpc2'

v1.45.3

What's Changed

• Bugfix: formula-path for Promoting to homebrew by @Benbentwo in #424

Full Changelog: v1.45.1...v1.45.3

v1.45.1

What's Changed

• Update Secret Reference to Fix Brew workflow by @Benbentwo in #423

Full Changelog: v1.45.0...v1.45.1

v1.45.0

what

• Add dependencies on external files and folders to settings.depends_on
• Fix context inheritance in Atmos imports with Go templates with Multiple inheritance
• Add/update docs

why

• Add dependencies on external files and folders to settings.depends_on. Allow specifying that an Atmos component depends on external files or folders. When the files or folders are modified, the command atmos describe affected will detect that and include the dependent component to the affected output list. This is used to specify dependencies on external files and folders to trigger the affected stacks in GitHub Actions and Spacelift

  Dependencies on external files (not in the component's folder) are defined using the file attribute in the settings.depends_on map. For example:

  components:
    terraform:
      top-level-component3:
        metadata:
          component: "top-level-component1"
        settings:
          depends_on:
            1:
              file: "examples/complete/components/terraform/mixins/introspection.mixin.tf"

  In the configuration above, we specify that the Atmos component top-level-component3 depends on the file
  examples/complete/components/terraform/mixins/introspection.mixin.tf (which is not in the component's folder). If the file gets modified, the component top-level-component3 will be included in the atmos describe affected command output. For example:

    [
      {
        "component": "top-level-component3",
        "component_type": "terraform",
        "component_path": "components/terraform/top-level-component1",
        "stack": "tenant1-ue2-test-1",
        "stack_slug": "tenant1-ue2-test-1-top-level-component3",
        "atlantis_project": "tenant1-ue2-test-1-top-level-component3",
        "affected": "file",
        "file": "examples/complete/components/terraform/mixins/introspection.mixin.tf"
      }
    ]

  Dependencies on external folders are defined using the folder attribute in the settings.depends_on map.
  For example:

  components:
    terraform:
      top-level-component3:
        metadata:
          component: "top-level-component1"
        settings:
          depends_on:
            1:
              file: "examples/complete/components/terraform/mixins/introspection.mixin.tf"
            2:
              folder: "examples/complete/components/helmfile/infra/infra-server"

  In the configuration above, we specify that the Atmos component top-level-component3 depends on the folder
  examples/complete/components/helmfile/infra/infra-server. If any file in the folder gets modified,
  the component top-level-component3 will be included in the atmos describe affected command output. For example:

    [
      {
        "component": "top-level-component3",
        "component_type": "terraform",
        "component_path": "components/terraform/top-level-component1",
        "stack": "tenant1-ue2-test-1",
        "stack_slug": "tenant1-ue2-test-1-top-level-component3",
        "atlantis_project": "tenant1-ue2-test-1-top-level-component3",
        "affected": "folder",
        "folder": "examples/complete/components/helmfile/infra/infra-server"
      }
    ]

• Fix context inheritance in Atmos imports with Go templates with Multiple inheritance. The multiple inheritance of the context in imports with hierarchical Go templates had an issue which was not detected by the tests. Now the hierarchical configurations like the following are working:

# examples/complete/stacks/catalog/terraform/service-iam-role/defaults.tmpl
components:
  terraform:
    service-iam-role/{{ .app_name }}/{{ .service_environment }}:
      metadata:
        component: infra/service-iam-role
      settings:
        spacelift:
          workspace_enabled: false
      vars:
        enabled: {{ .enabled }}
        tags:
          Service: {{ .app_name }}
        service_account_name: {{ .app_name }}
        service_account_namespace: {{ .service_account_namespace }}
        {{ if hasKey . "iam_managed_policy_arns" }}
        iam_managed_policy_arns:
          {{ range $i, $iam_managed_policy_arn := .iam_managed_policy_arns }}
          - '{{ $iam_managed_policy_arn }}'
          {{ end }}
        {{ end }}
        {{ if hasKey . "iam_source_policy_documents" }}
        iam_source_policy_documents:
          {{ range $i, $iam_source_policy_document := .iam_source_policy_documents }}
          - '{{ $iam_source_policy_document }}'
          {{ end }}
        {{ end }}

# examples/complete/stacks/catalog/terraform/service-iam-role/webservices/defaults.tmpl
import:
  - path: catalog/terraform/service-iam-role/defaults.tmpl
    context:
      enabled: true
      app_name: webservices
      iam_managed_policy_arns:
        - arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
        - arn:aws:iam::aws:policy/AmazonKinesisFullAccess

# examples/complete/stacks/catalog/terraform/service-iam-role/webservices/prod.defaults.tmpl
import:
  - path: catalog/terraform/service-iam-role/webservices/defaults.tmpl
    context:
      iam_managed_policy_arns:
        - arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
        - arn:aws:iam::aws:policy/AmazonKinesisFullAccess
        - arn:aws:iam::aws:policy/AmazonS3FullAccess
        - arn:aws:iam::aws:policy/AmazonSNSFullAccess
        - arn:aws:iam::aws:policy/AmazonSQSFullAccess
        - arn:aws:iam::aws:policy/AmazonECS_FullAccess
        - arn:aws:iam::aws:policy/CloudWatchFullAccess
        - arn:aws:iam::aws:policy/AWSDataPipeline_FullAccess
        - arn:aws:iam::aws:policy/CloudFrontFullAccess

# examples/complete/stacks/catalog/terraform/service-iam-role/webservices/prod.yaml
import:
  - path: catalog/terraform/service-iam-role/webservices/prod.defaults.tmpl
    context:
      service_account_namespace: prod
      service_environment: prod

# examples/complete/stacks/orgs/cp/tenant2/prod/us-east-2.yaml
import:
  - catalog/terraform/service-iam-role/webservices/prod

atmos describe component service-iam-role/webservices/prod -s tenant2-ue2-prod

The iam_managed_policy_arns variable is correctly inherited from examples/complete/stacks/catalog/terraform/service-iam-role/webservices/prod.defaults.tmpl

v1.44.0

what

• Update Spacelift deps
• Update atmos validate stacks command
• Update docs
• Update tests

why

• Update Spacelift deps - use the new settings.depends_on schema (introduced for the atmos describe dependents command) to specify stack dependencies instead of the deprecated settings.spacelift.depends_on schema (note that the old settings.spacelift.depends_on schema is still supported for backwards compatibility)

• NOTE: This is a temporary fix to prevent the errors, in a follow up PR, we'll update the command to correctly evaluate all Go templates in the imported file names.

  Update atmos validate stacks command - if an imported file is a Go template, don't count it as an invalid import. This prevents errors like this:

atmos validate stacks

no matches found for the import 'orgs/cp/{{ .tenant }}/{{ .stage }}/_defaults' 
in the file 'catalog/terraform/eks_cluster_tmpl_hierarchical.yaml'

v1.43.0

what

Add the --skip-lock-file flag to the atmos terraform clean command

why

For users who want to keep a stable lock file but otherwise want to clean the component directory

v1.42.0

what

• Add Sprig functions to Atmos Go templates in imports
• Various fixes and improvements for atmos describe component and component dependencies calculation
• Update docs https://atmos.tools/cli/commands/describe/component/

why

• Sprig functions in Atmos Go templates in imports provide over 70 useful functions for Go’s template language

• Atmos component validation (using atmos validate component command and OPA policies in YAML) now has access to all the outputs that the atmos describe component generates (before, some recently added output were not present to the OPA policies for validation)

• Update and improve component dependencies calculation

The command atmos describe component now outputs three different dependency-related attributes:

• imports - a list of all imports in the Atmos stack (this shows all imports in the stack, related to the component and not)

• deps_all - a list of all component stack dependencies (stack config files where the component settings are defined, either inline or via imports)

• deps - a list of component stack dependencies where the final values of all component configurations are defined
  (after the deep-merging and processing all the inheritance chains and all the base components)

deps:
- catalog/terraform/base-component-1
- catalog/terraform/base-component-4
- catalog/terraform/derived-component-3
- catalog/terraform/spacelift-and-backend-override-1
- mixins/region/us-east-2
- mixins/stage/test1
- orgs/cp/_defaults
- orgs/cp/tenant1/_defaults
deps_all:
- catalog/terraform/base-component-1
- catalog/terraform/base-component-3
- catalog/terraform/base-component-4
- catalog/terraform/derived-component-3
- catalog/terraform/spacelift-and-backend-override-1
- mixins/region/us-east-2
- mixins/stage/test1
- orgs/cp/_defaults
- orgs/cp/tenant1/_defaults
- orgs/cp/tenant1/test1/us-east-2
imports:
- catalog/helmfile/echo-server
- catalog/helmfile/infra-server
- catalog/helmfile/infra-server-override
- catalog/terraform/base-component-1
- catalog/terraform/base-component-2
- catalog/terraform/base-component-3
- catalog/terraform/base-component-4
- catalog/terraform/derived-component-1
- catalog/terraform/derived-component-2
- catalog/terraform/derived-component-3
- catalog/terraform/mixins/test-1
- catalog/terraform/mixins/test-2
- catalog/terraform/services/service-1
- catalog/terraform/services/service-1-override
- catalog/terraform/services/service-1-override-2
- catalog/terraform/services/service-2
- catalog/terraform/services/service-2-override
- catalog/terraform/services/service-2-override-2
- catalog/terraform/services/top-level-service-1
- catalog/terraform/services/top-level-service-2
- catalog/terraform/spacelift-and-backend-override-1
- catalog/terraform/test-component
- catalog/terraform/test-component-2
- catalog/terraform/test-component-override
- catalog/terraform/test-component-override-2
- catalog/terraform/test-component-override-3
- catalog/terraform/top-level-component1
- catalog/terraform/top-level-component2
- catalog/terraform/vpc
- mixins/region/us-east-2
- mixins/stage/test1
- orgs/cp/_defaults
- orgs/cp/tenant1/_defaults
- orgs/cp/tenant1/test1/_defaults

The difference between the imports, deps_all and deps outputs is as follows:

• imports shows all imports in the stack for all components. This can be useful in GitHub actions (or other CI/CD systems) and in OPA validation policies to check whether an import is allowed in the stack or not

• deps_all shows all component stack dependencies (imports and root-level stacks) where any configuration for the component is present. This also can be useful in CI/CD systems and OPA validation policies to check whether a user or a team is allowed to import a particular config file for the component in the stack

• deps shows all the component stack dependencies where the FINAL values from all the component sections are defined (after the deep-merging and processing all the inheritance chains and all the base components). This is useful in CI/CD systems (e.g. Spacelift) to detect all the affected files that the component depends on (and trigger the component's stack if any of the files is affected). deps is usually a much smaller list than deps_all and can differ from it in the following ways:

  • The component can inherit configurations from many base components, see Component Inheritance, and import those base component configurations

  • The component can override all the default variables from the base components, and the final values are not dependent on the base components config anymore. For example, derived-component-3 import the base component base-component-4 configuration, inherits from it, and overrides all the variables:

  # Import the base component config
  import:
    - catalog/terraform/base-component-4
  
  components:
    terraform:
      derived-component-3:
        metadata:
          component: "test/test-component"  # Point to the Terraform component
          inherits:
            # Inherit all the values from the base component
            - base-component-4
        vars:
          # Override all the variables from the base component

  • Atmos detects that and does not include the base component base-component-4 config file into the deps output since the component does not directly depend on it (all values are coming from derived-component-3). This will help, for example, to not trigger the component's Spacelift stack unnecessary if only the base-component-4 changes, preventing the unrelated stack runs

  • In the above case, the deps_all output will include both derived-component-3 and base-component-4, but the deps output will only include derived-component-3

NOTE: Before this update, we had only the deps output (and used it in Spacelift policies to detect affected files and trigger the stacks), which was somewhere in between the new deps and deps_all outputs. It showed too much and too little at the same time:

• After we introduced multiple and multi-level component inheritance (see https://atmos.tools/core-concepts/components/inheritance), the deps output was not updated to check for the base components that were completely overridden by the derived component, and all those base components were included in the output, triggering Spacelift stacks unnecessary

• Atmos did not check the cases where only imports were used to import the component config from the catalog w/o having anything related to the component defined inline in other config files. Atmos was including all those YAML config files into the deps output, and if they were updated (e.g. by importing a completely different component into the stack), triggering the component's Spacelift stack unnecessary. Now all of that is added only to the deps_all output having the deps output much smaller

• Now we are using the Sources of component variables and other sections to find out all the dependencies for the deps and deps_all outputs