v1.3.17

🚀 Enhancements

Add multiple inheritance and mixins for components. Add `metadata.component` and `metadata.inherits` settings @aknysh (#101)

what

• Add multiple inheritance (and multiple inheritance chains) and mixins for components
• Add metadata.component and metadata.inherits settings

why

• Allow a component to inherit from many base components or mixins, each base component having its own inheritance chain, effectively making it an inheritance matrix. It uses a method similar to Method Resolution Order (MRO), which is how Python supports multiple inheritance.

• metadata.component points to the component implementation (e.g. in components/terraform folder).
  It does not specify inheritance. It overrides the deprecated top-level component attribute, which performed two roles: 1) point to a component implementation; 2) specify a base component to inherit from (only one) .

• metadata.inherits is a list of component or mixins names from which the current component inherits. In the case of multiple base components, it is processed left to right, in the order by which it was declared.
  For example: metadata.inherit: [componentA, componentB] will deep-merge all the base components of componentA (each component overriding its base), then all the base components of componentB (each component overriding its base), then the two results are deep-merged together (componentB inheritance chain will override values from `componentA' inheritance chain).

related

• In object-oriented programming languages, a mixin is a class that contains methods for use by other classes without having to be the parent class of those other classes

test

Given this component config

import:
  - catalog/terraform/mixins/test-*.*

components:
  terraform:
    "test/test-component-override-3":
      settings:
        spacelift:
          workspace_enabled: false
      vars: {}
      env:
        TEST_ENV_VAR1: "val1-override-3"
        TEST_ENV_VAR2: "val2-override-3"
        TEST_ENV_VAR3: "val3-override-3"
        TEST_ENV_VAR4: "val4-override-3"
      metadata:
        # `real` is implicit, you don't need to specify it; `abstract` makes the component protected from being deployed
        type: real
        # Terraform component. Must exist in `components/terraform` folder.
        # If not specified, it's assumed that this component `test/test-component-override-3` is also a Terraform component
        # in `components/terraform/test/test-component-override-3` folder
        component: "test/test-component"
        # Multiple inheritance. It's a down-top/left-right matrix (similar to Method Resolution Order (MRO), which is how Python supports multiple inheritance).
        # All base components and mixins are processed and deep-merged in the order they are specified in the `inherits` list:
        # 1. `test/test-component-override-2` overrides `test/test-component-override` and its base components (all the way up its inheritance chain).
        # 2. `mixin/test-1` overrides `test/test-component-override-2` and its base components (all the way up its inheritance chain).
        # 3. `mixin/test-2` overrides `mixin/test-1` and its base components (all the way up its inheritance chain).
        # 4. This `test/test-component-override-3` component overrides `mixin/test-2` and its base components (all the way up its inheritance chain).
        # Inheritance:  test/test-component-override-3 -> mixin/test-2 -> mixin/test-1 -> test/test-component-override-2 -> test/test-component-override -> test/test-component
        inherits:
          - "test/test-component-override"
          - "test/test-component-override-2"
          - "mixin/test-1"
          - "mixin/test-2"

atmos terraform plan test/test-component-override-3 -s=tenant1-ue2-dev

Results in

Command info:
Terraform binary: terraform
Terraform command: plan
Arguments and flags: []
Component: test/test-component-override-3
Terraform component: test/test-component
Inheritance:  test/test-component-override-3 -> mixin/test-2 -> mixin/test-1 -> test/test-component-override-2 -> test/test-component-override -> test/test-component
Stack: tenant1/ue2/dev
Working dir: examples/complete/components/terraform/test/test-component

Executing command:
/usr/local/bin/terraform plan -var-file tenant1-ue2-dev-test-component-override-3.terraform.tfvars.json -out tenant1-ue2-dev-test-component-override-3.planfile

Changes to Outputs:
  + service_1_id = "eg-ue2-dev-mixin-2"
  + service_2_id = "eg-ue2-dev-service-2-override-2"

v1.3.16

🚀 Enhancements

Add `base_path` setting to `atmos` config @aknysh (#100)

what

• Add base_path setting to atmos config
• Add atmos logos
• Update README

why

• Provide/configure a base path for all components and stacks
• Allow controlling stacks and components paths using only one setting or ENV var
• Can also be set using ATMOS_BASE_PATH ENV var, or --base-path command-line argument
• Supports both absolute and relative paths
• If not provided or is an empty string, components.terraform.base_path, components.helmfile.base_path and stacks.base_pathare independent settings (supporting both absolute and relative paths)
• If base_path is provided (via ENV var or command-line arg), components.terraform.base_path, components.helmfile.base_path and stacks.base_path are considered paths relative to base_path

test

ATMOS_BASE_PATH=./examples/complete atmos terraform apply test/test-component -s=tenant1-ue2-dev

Found ENV var ATMOS_BASE_PATH=./examples/complete

Variables for the component 'test/test-component' in the stack 'tenant1/ue2/dev':

enabled: true
environment: ue2
namespace: eg
region: us-east-2
service_1_name: service-1
service_2_name: service-2
stage: dev
tenant: tenant1

Writing variables to file:
examples/complete/components/terraform/test/test-component/tenant1-ue2-dev-test-component.terraform.tfvars.json

v1.3.15

🚀 Enhancements

Add `metadata` top-level section to all components. Add `type` attribute to the `metadata` section (`metadata.type`). Update README @aknysh (#99)

what

• Update README
• Add metadata top-level section to all components
• Add type attribute to the metadata section (metadata.type)

why

• Remove the old/obsolete sections from README (new README with new atmos features coming soon)

• The metadata section is used to describe metadata for a component (it's for the component only, not deep-merged with base components or globals). Other attributes might be added to the section in the future to describe component types, kinds and different behaviors

• type attribute in the metadata section specifies the component type: real (implicit and default, and can be provisioned) or abstract (non-deployable, just a base class for derived components, like abstract classes in OOP)

components:
  terraform:
    "test/test-component":
      settings:
        spacelift:
          workspace_enabled: true
      # Setting `metadata.type: abstract` makes the component `abstract` (similar to OOP abstract classes, which can't be instantiated),
      # explicitly prohibiting the component from being deployed,
      # and a Spacelift stack from being created for the component (even if `settings.spacelift.workspace_enabled: true`).
      # `terraform apply` and `terraform deploy` will fail with an error that the component cannot be provisioned.
      # All other terraform commands on this component will succeed.
      # If `metadata.type` attribute is not specified, it defaults to `real` (which means the component can be provisioned).
      metadata:
        type: real # `real` is implicit, you don't need to specify it; `abstract` makes the component protected from being deployed

• metadata.type attribute in the metadata section allows explicitly disabling a component from being deployed. This is useful for non-deployable base components (which are just blueprints for derived components) if we need to prohibit the base components from being deployed (accidentally via CLI or via Spacelift).
  For Spacelift, this will prevent derived components from inheriting settings.spacelift.workspace_enabled=false of not-deployable component (allowing specifying settings.spacelift.workspace_enabled=true` at a global level making the config DRY)

test

components:
  terraform:
    "test/test-component":
      settings:
        spacelift:
          workspace_enabled: false
      metadata:
        type: abstract

atmos terraform apply test/test-component -s=tenant1-ue2-dev  

Variables for the component 'test/test-component' in the stack 'tenant1/ue2/dev':

enabled: true
environment: ue2
namespace: eg
region: us-east-2
service_1_name: service-1
service_2_name: service-2
stage: dev
tenant: tenant1

Abstract component 'test/test-component' cannot be provisioned 
since it's explicitly prohibited from being deployed with 'metadata.type: abstract' attribute

Spacelift config

settings:
  spacelift:
    workspace_enabled: false

atmos describe component test/test-component -s=tenant1-ue2-dev

results in:

settings:
  spacelift: {}

v1.3.14

🚀 Enhancements

Auto generate backend key based on component name for azurerm @lukeorellana (#95)

what

When using the AzureRM backend type, automatically generate backend key based on component name if the backend key is not specified at the component level:

• Key is created in {Global Backend Key value}/{component name}.terraform.tfstate format. Places all component states in a storage account folder with the name of the global backend key value.
• If the key is specified at the component level, the backend key is not automatically generated.

Before:

terraform:
  backend_type: azurerm
  backend:
    azurerm:
      subscription_id: 88888-8888-8888-8888-8888888888
      resource_group_name: rg-terraform-state
      storage_account_name: staterraformstate
      container_name: dev-tfstate

components:
  terraform:
    "monitoring":
      backend:
          azurerm:
            key: dev.atmos/monitoring.terraform.tfstate
        
    "networking":
      backend:
          azurerm:
            key: dev.atmos/networking.terraform.tfstate

After:

terraform:
  backend_type: azurerm
  backend:
    azurerm:
      subscription_id: 88888-8888-8888-8888-8888888888
      resource_group_name: rg-terraform-state
      storage_account_name: staterraformstate
      container_name: dev-tfstate
      key: dev.atmos

components:
  terraform:
    "rg": {}

    "monitoring": {}

    "vault": {}

backend.tf:

{
  "terraform": {
    "backend": {
      "azurerm": {
        "subscription_id": "88888-8888-8888-8888-8888888888",
        "container_name": "dev-tfstate",
        "key": "dev.atmos/rg.terraform.tfstate",
        "resource_group_name": "rg-terraform-state",
        "storage_account_name": "staterraformstate"
      }
    }
  }
}

why

The (azure backend) does not currently have a prefix option for workspaces. When using Atmos with Azure, users are forced to specify each backend key for every component.

references

hashicorp/terraform#28985

v1.3.13

🚀 Enhancements

Improve `terraform import`. Add help for all `atmos` commands @aknysh (#94)

what

• Improve terraform import
• Add help for all atmos commands

why

• Since terraform import command requires a region, atmos terraform import looks for region in the variables for the specified component and stack, and if it finds it, sets AWS_REGION=<region> ENV var before executing the command

• Help needed for all atmos commands and subcommands

related

• Closes #92
• Closes #74

test

atmos --help

atmos terraform --help

atmos terraform plan --help

atmos helmfile --help

atmos helmfile diff --help

v1.3.12

🚀 Enhancements

Update/fix component inheritance chain @aknysh (#87)

what

• Update/fix component inheritance chain

why

• Add component inheritance chain for helmfile (it was not used before)

• Fix these errors for some YAML stack configs

Error: Terraform component 'aws-backup-common' defines attribute 'component: aws-backup', 
but `aws-backup' is not defined in the stack '/atmos_root/stacks/catalog/aws-backup/common.yaml'

• Fix these errors for some YAML stack configs

runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc0206404e0 stack=[0xc020640000, 0xc040640000]
fatal error: stack overflow

🐛 Bug Fixes

Update/fix component inheritance chain @aknysh (#87)

what

• Update/fix component inheritance chain

why

• Add component inheritance chain for helmfile (it was not used before)

• Fix these errors for some YAML stack configs

Error: Terraform component 'aws-backup-common' defines attribute 'component: aws-backup', 
but `aws-backup' is not defined in the stack '/atmos_root/stacks/catalog/aws-backup/common.yaml'

• Fix these errors for some YAML stack configs

runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc0206404e0 stack=[0xc020640000, 0xc040640000]
fatal error: stack overflow

v1.3.11

🚀 Enhancements

Disable component inheritance chain @aknysh (#85)

what

• Disable component inheritance chain for now to make all stacks work

why

• While it's working in all the tests, and in 99% of the real infra stacks, some edge cases of stack config are not handled correctly and cause gorotine panic
• All edge cases will be tested and fixed in the consecutive PR

v1.3.10

New actions to bump homebrew and auto format action @nitrocode (#81)

what

• Added bumping homebrew formula automatically
• Added auto-format workflow
• Addititional install instructions

why

• Keep homebrew version consistent with this repo's releases
• Generate new README.md from yaml
• Easy install instructions

references

• https://github.com/marketplace/actions/bump-homebrew-formula
• https://raw.githubusercontent.com/cloudposse/terraform-null-label/master/.github/workflows/auto-format.yml
• Previous PR #80

Add install instructions @nitrocode (#80)

what

• add install instructions

why

• easy install instructions

references

• Closes #79
• https://formulae.brew.sh/formula/atmos

🚀 Enhancements

Component inheritance chain @aknysh (#84)

what

• Add component inheritance chain (unlimited levels)
• Add inheritance section to component outputs
• Show inheritance chain when executing atmos commands (Inheritance: test/test-component-override-2 -> test/test-component-override -> test/test-component)

why

• Allow to inherit config from more than one component, e.g. Component A inherits Component B which in turn inherits Component C (each one can override any attributes)
• Reuse component config
• DRY config

related

• Closes #82

test

atmos terraform plan test/test-component-override-2 -s tenant1-ue2-dev

Command info:
Terraform binary: /usr/local/bin/terraform
Terraform command: plan
Arguments and flags: []
Component: test/test-component-override-2
Base component: test/test-component
Inheritance: test/test-component-override-2 -> test/test-component-override -> test/test-component
Stack: tenant1/ue2/dev

test/test-component-override-2:
  component: test/test-component
  inheritance:
  - test/test-component-override
  - test/test-component

v1.3.9

🐛 Bug Fixes

Change misspelled error messages @johncblandii (#78)

what

• Fixed misspelling in error message

why

• Component 'eks' does not exixt in /atmos_root/components/terraform

references

v1.3.8

🚀 Enhancements

Detect ENV vars in YAML stack config and set them for command execution. Make `workspace_key_prefix` config DRY @aknysh (#77)

what

• Detect ENV vars in YAML stack config and set them for command execution
• Make workspace_key_prefix config DRY
• Don't delete the generated terraform varfiles after each command
• Update tests
• General cleanup

why

• Detect ENV vars in YAML stack config and set them for command execution - allow specifying ENV vars in YAML config files. For each component in a stack, ENV vars are deep-merged in this order: global ENV vars, terraform section ENV vars, base component ENV vars, component ENV vars. The when commands are executed, print the final set of ENV vars and set them in the executing shell. This will allow controlling the tools behavior (e.g. Terraform, helmfile) by YAML configs

Command info:
Terraform binary: /usr/local/bin/terraform
Terraform command: plan
Arguments and flags: []
Component: test/test-component-override
Base component: test/test-component
Stack: tenant1/ue2/dev
Working dir: ./examples/complete/components/terraform/test/test-component

Using ENV vars:
TEST_ENV_VAR3=val3-override
TEST_ENV_VAR1=val1-override
TEST_ENV_VAR2=val2
TEST_ENV_VAR4=val4

Executing command:
/usr/local/bin/terraform workspace select tenant1-ue2-dev-test-component-override

Executing command:
/usr/local/bin/terraform plan -var-file tenant1-ue2-dev-test-component-override.terraform.tfvars.json -out tenant1-ue2-dev-test-component-override.planfile

• Check if backend section has workspace_key_prefix for s3 backend type. If it does not, use the component name instead. It also propagates to remote_state_backend section of s3 type. This will allow to have components catalog files DRY without repeating the same config (which now can be generated automatically by the component names). Components folders are supported and taken into account in the generated workspace_key_prefix. This can be overridden as before per component.

This config

components:
  terraform:
    "test/test-component":
      backend:
        s3:
          workspace_key_prefix: test-test-component
      settings:
        spacelift:
          workspace_enabled: true

is now the same as this

components:
  terraform:
    "test/test-component":
      settings:
        spacelift:
          workspace_enabled: true

and produces the same result (confirmed by the updated tests)

backend:
  workspace_key_prefix: test-test-component
backend_type: s3
remote_state_backend:
  workspace_key_prefix: test-test-component
remote_state_backend_type: s3

• Don't delete the generated terraform varfiles after each command - sometimes it's useful to be able to run atmos terraform plan and then use the generated varfile and planfile in other terraform commands. The varfile and planfile can be deleted by now supported command tmos terraform clean <component> -s <stack>