🏗️ terraform-aylas-servers

Terraform and Ansible artifacts to manage the Aylas Community server infrastructure configuration, which includes a Minecraft server hosted on Always Free Oracle Cloud Infrastructure instances.

This repository contains the definition files for the production Aylas Community server infrastructure configuration, following an Infrastructure as Code (IaC) approach.

With this approach, we aim to follow established technical best practices and improve the consistency, reliability, traceability, deployment speed and disaster recovery procedures of our infrastructure compared to manual processes. In addition, we seek to to bring our development and server management forces closer together, fostering DevOps-inspired workflows.

We are releasing our configuration to the public to provide insight into what makes our community tick, and potentially inspire others. Feel free to suggest improvements, or use parts of our configuration in your own deployments!

✨ Highlights

• 🏗️ Leverages Terraform and Ansible to manage our Oracle Cloud Infrastructure from the sign-up on OCI to the moment our services are up, with minimal manual intervention.
• 🖥️ Sets up a 24/7 Purpur Minecraft server using the recommended brucethemoose's state of the art JVM flags on a minimal, standard ARM Ubuntu 22.04 virtual machine (a.k.a. instance). Oracle GraalVM (formerly Oracle GraalVM Enterprise Edition) is used, leveraging its top-tier compiler based on JVMCI, which generates significantly better optimized code than OpenJDK's C2 for some workloads.
• 💾 Automated 3-2-1 backup strategy powered by duplicity:
  • We store the live data, a local backup collection and a remote backup collection on MEGA.
  • Backup tarballs are compressed with Gzip, encrypted with a symmetric cipher using GPG, and usable even if slightly corrupted thanks to redundant PAR2 archives.
  • The server is automatically restarted every few days to do an incremental backup of its files. Before updating to a new Minecraft version, the incremental backups are deleted and a full backup is made.
  • A notification is sent to a Discord channel via webhooks when a backup is done. (This can be disabled by not defining the webhook URL in the Ansible playbooks.)
• 👌 Easy, user-friendly and secure remote management over SSH:
  • Server applications (for now, the Purpur server) run on a dedicated and unprivileged user, with SSH forwarding and SFTP access disabled. Shell access is restricted to a locked-down tmux session that only allows interaction with the server console, providing an experience similar to interacting with a local server console window. Unexpected critical server files modifications are prevented via the immutable filesystem attribute.
  • A secondary account with only SFTP access is provided to manage server files. These SFTP sessions are jailed to the server directory, so that they see a clean directory hierarchy free from system files. The directory can be mounted as a filesystem for quick access from Linux, Windows, BSD and macOS clients by using rclone, SSHFS, or SSHFS-Win.
  • The server console is extended with a server controller that handles interactively starting a server after it stops, restoring backups, and updating or backing up the server files.
  • Secure SSH login keys are automatically generated, set up and copied to the local login_keys directory during initial provisioning. Password authentication is disabled.
  • We make it easy to change the SSH server port during deployment, which is an effective defense against the log spam and waste of CPU cycles caused by the mass SSH scans that plague the Internet with a minimal impact on usability and availability.
• 🛡️ Game server hardening based on AppArmor, which implements mandatory access control mechanisms used to prevent unexpected server behavior in an easy-to-manage manner.
• 🆕 Automated daily system and server software updates.
• 🚨 The ttyAMA0 serial console is set up for out-of-band management, ensuring that trusted parties can control the instance using the OCI web panel even if SSH connectivity is lost.
• 📊 Per-service disk quotas to guarantee the expected distribution of disk space and mitigate the impact of errant applications.
• 🌐 IPv6 support for future-proofing and letting every client use the IP stack version that works best for them.
• 🔝 Static analysis checks are executed on each push by GitHub Actions runners to guarantee code quality.
• 🙌 Due to the usage of vendor-agnostic IaC tools and standard Linux software, we avoid cloud vendor lock-in. It should be easy to port this configuration to other cloud environments.
• 🤑 Thanks to OCI's generous Always Free tier, you can get all of this for free!*

^{_{* If you are over legal age, and willing to give Oracle accurate contact information and your credit card number during registration. Virtual cards are not accepted. Always Free resources may not be always available in all regions, and they can only be deployed to your home region, which is chosen when signing up and can't be changed. Oracle is evil may change these conditions in the future. We don't expect to contact their support for anything. Abusive usage of these resources may cause account termination. Apart from than that, we're not aware of any other fine print in their conditions.}}

📥 Getting started

After cloning the repository, run terrashell.sh on a Linux-like environment with python3 and pip available (Linux, WSL) to set up a temporary virtual environment where Terraform, Ansible and the OCI CLI will be installed. This environment is automatically cleaned up on exit. If you are using another operating system, you can set up this environment manually. From here you can run terraform apply to deploy and provision the infrastructure in less than 10 minutes, in addition to other Terraform and Ansible commands.

✏️ Variables and secrets

For obvious reasons, the default values of some variables may not be suitable for your particular scenario (i.e., the OCI tenancy region), and some pieces of configuration are stored as sensitive Terraform variables or Ansible Vault encrypted variables. These variables are defined at the *.ansible.yml and variables.tf files. You must review these variables prior to your first deployment and change their values accordingly.

⚙️ Development environment

We recommend using Visual Studio Code with the extensions recommended by this repository because they provide a good development experience, with code analysis and coding assistance features (lints, autocompletion...). However, you can use other text editors if you wish.

📁 Project structure

main.tf is the entrypoint for the infrastructure definition, which invokes Ansible playbooks (files with .ansible.yml extension) to provision instances with the specific software configuration. The playbooks leverage reusable roles defined at the roles directory, which execute related configuration tasks. As usual with Ansible, it is possible to run playbooks on already provisioned machines to apply new configurations.

✨ Contributing

Pull requests are accepted. Feel free to contribute if you can improve some aspect of our server infrastructure!

⚖️ License

MIT © Alejandro González