Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Terraform with OpenTofu #69

Merged
merged 1 commit into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 19 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ In particular, this project aims to provide the following benefits to Open edX o

## Technology stack and architecture

1. At the base is a Kubernetes cluster, which you must provide (e.g. using Terraform to provision Amazon EKS).
* Any cloud provider such as AWS or Digital Ocean should work. There are Terraform examples in the `infra-examples` folder but it is just a starting point and not recommended for production use.
1. At the base is a Kubernetes cluster, which you must provide (e.g. using OpenTofu to provision Amazon EKS).
* Any cloud provider such as AWS or Digital Ocean should work. There are OpenTofu examples in the `infra-examples` folder but it is just a starting point and not recommended for production use.
2. On top of that, this project's helm chart will install the shared resources you need - an ingress controller, monitoring, database clusters, etc. The following are included but can be disabled/replaced if you prefer an alternative:
* Ingress controller: [ingress-nginx](https://kubernetes.github.io/ingress-nginx/)
* Automatic HTTPS cert provisioning: [cert-manager](https://cert-manager.io/)
* Autoscaling: `metrics-server` and `vertical-pod-autoscaler`
* Search index: ElasticSearch (support for OpenSearch is planned)
* Monitoring: TODO
* Database clusters: TODO (for now we recommend provisioning managed MySQL/MongoDB database clusters from your cloud provider using Terraform or a tool like [Grove](https://grove.opencraft.com/).)
* Database clusters: TODO (for now we recommend provisioning managed MySQL/MongoDB database clusters from your cloud provider using OpenTofu or a tool like [Grove](https://grove.opencraft.com/).)
* Where possible, we try to configure these systems to **auto-detect** newly deployed Open edX instances and adapt to them automatically; where that isn't possible, Tutor plugins are used so that the instances self-register or self-provision the shared resources as needed.
3. [Tutor](https://docs.tutor.overhang.io/) is used to build the container images that will be deployed onto the cluster.
* This project's Tutor plugin is required to make the images compatible with the shared resources deployed by the Helm chart.
Expand Down Expand Up @@ -100,22 +100,22 @@ infrastructure examples included in this repo for such purposes.

* An aws account id
* Kubectl 1.27
* Terraform 1.5.x or higher
* OpenTofu 1.6.x or higher
* Helm

1. Clone this repository and navigate to `./infra-examples/aws`. You'll find Terraform modules for `vpc` and `k8s-cluster`
1. Clone this repository and navigate to `./infra-examples/aws`. You'll find OpenTofu modules for `vpc` and `k8s-cluster`
resources. Proceed creating the `vpc` resources first, followed by the `k8s-cluster` resources. Make sure to have the target
AWS account ID available, and then execute the following commands on every folder:

```sh
terraform init
terraform plan
terraform apply -auto-approve
tofu init
tofu plan
tofu apply -auto-approve
```

It will create an EKS cluster in the new VPC. Required Karpenter resources will also be created.

2. Once the `k8s-cluster` is created, run the `terraform output` command on that module and copy the following output variables:
2. Once the `k8s-cluster` is created, run the `tofu output` command on that module and copy the following output variables:

* cluster_name
* karpenter_irsa_role_arn
Expand Down Expand Up @@ -312,7 +312,7 @@ Just run `helm uninstall --namespace harmony harmony` to uninstall this.

### How to create a cluster for testing on DigitalOcean

If you use DigitalOcean, you can use Terraform to quickly spin up a cluster, try this out, then shut it down again.
If you use DigitalOcean, you can use OpenTofu to quickly spin up a cluster, try this out, then shut it down again.

Here's how. First, put the following into `infra-examples/digitalocean/secrets.auto.tfvars` including a valid DigitalOcean access token:

Expand All @@ -325,18 +325,18 @@ Then run:

```sh
cd infra-examples/digitalocean
terraform init
terraform apply
tofu init
tofu apply
cd ..
export KUBECONFIG=`pwd`/infra-examples/digitalocean/kubeconfig
```

Then follow steps 1-4 above. When you're done, run `terraform destroy` to clean
Then follow steps 1-4 above. When you're done, run `tofu destroy` to clean
up everything.

## Appendix C: how to create a cluster for testing on AWS

Similarly, if you use AWS, you can use Terraform to spin up a cluster, try this out, then shut it down again.
Similarly, if you use AWS, you can use OpenTofu to spin up a cluster, try this out, then shut it down again.
Here's how. First, put the following into `infra-examples/aws/vpc/secrets.auto.tfvars` and `infra-examples/aws/k8s-cluster/secrets.auto.tfvars`:

```terraform
Expand All @@ -350,14 +350,14 @@ Then run:
```bash
aws sts get-caller-identity # to verify that awscli is properly configured
cd infra-examples/aws/vpc
terraform init
terraform apply # run time is approximately 1 minute
tofu init
tofu apply # run time is approximately 1 minute
cd ../k8s-cluster
terraform init
terraform apply # run time is approximately 30 minutes
tofu init
tofu apply # run time is approximately 30 minutes

# to configure kubectl
aws eks --region us-east-1 update-kubeconfig --name tutor-multi-test --alias tutor-multi-test
```

Then follow steps 1-4 above. When you're done, run `terraform destroy` in both the `aws` and `k8s-cluster` modules to clean up everything.
Then follow steps 1-4 above. When you're done, run `tofu destroy` in both the `aws` and `k8s-cluster` modules to clean up everything.
2 changes: 1 addition & 1 deletion infra-examples/aws/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Reference Architecture for AWS

This module includes Terraform modules to create AWS reference resources that are preconfigured to support Open edX as well as [Karpenter](https://karpenter.sh/) for management of [AWS EC2 spot-priced](https://aws.amazon.com/ec2/spot/) compute nodes and enhanced pod bin packing.
This module includes OpenTofu modules to create AWS reference resources that are preconfigured to support Open edX as well as [Karpenter](https://karpenter.sh/) for management of [AWS EC2 spot-priced](https://aws.amazon.com/ec2/spot/) compute nodes and enhanced pod bin packing.

## Virtual Private Cloud (VPC)

Expand Down
10 changes: 5 additions & 5 deletions infra-examples/aws/k8s-cluster/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ locals {
partition = data.aws_partition.current.partition

tags = {
"Name" = var.name
"openedx-k8s-harmony/name" = var.name
"openedx-k8s-harmony/region" = var.aws_region
"openedx-k8s-harmony/terraform" = "true"
"Name" = var.name
"openedx-k8s-harmony/name" = var.name
"openedx-k8s-harmony/region" = var.aws_region
"openedx-k8s-harmony/tofu" = "true"
}

}
Expand Down Expand Up @@ -168,7 +168,7 @@ module "eks" {
# See more details in
# https://github.com/terraform-aws-modules/terraform-aws-eks/blob/v19.16.0/modules/karpenter/README.md#external-node-iam-role-default
module "karpenter" {
source = "terraform-aws-modules/eks/aws//modules/karpenter"
source = "terraform-aws-modules/eks/aws/modules/karpenter"
version = "~> 19.16"

cluster_name = module.eks.cluster_name
Expand Down
8 changes: 4 additions & 4 deletions infra-examples/aws/vpc/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,10 @@ locals {
}

tags = {
"Name" = var.name
"openedx-k8s-harmony/name" = var.name
"openedx-k8s-harmony/region" = var.aws_region
"openedx-k8s-harmony/terraform" = "true"
"Name" = var.name
"openedx-k8s-harmony/name" = var.name
"openedx-k8s-harmony/region" = var.aws_region
"openedx-k8s-harmony/tofu" = "true"
}


Expand Down
4 changes: 2 additions & 2 deletions infra-examples/digitalocean/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ data "digitalocean_kubernetes_cluster" "cluster" {
name = var.cluster_name
# Set the depends_on so that the data source doesn't
# try to read from a cluster that doesn't exist, causing
# failures when trying to run a `terraform plan`.
# failures when trying to run a `tofu plan`.
depends_on = [module.k8s_cluster.cluster_id]
}

Expand Down Expand Up @@ -76,7 +76,7 @@ provider "kubectl" {
}


# Declare the kubeconfig as an output - access it anytime with "terraform output -raw kubeconfig"
# Declare the kubeconfig as an output - access it anytime with "tofu output -raw kubeconfig"
output "kubeconfig" {
value = module.k8s_cluster.kubeconfig.raw_config
sensitive = true
Expand Down