[#246] Prepare a base image for backend to speed up current process of building application for deployment #274
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
Introduces a new Dockerfile.base within the govtool/backend directory, aimed at creating a base image for the Haskell backend project. This base image includes all necessary pre-compiled dependencies, following the initiative to expedite the build process for deployment. The Dockerfile is based on the `haskell:9.2-buster` image and sets up an environment optimized for the GovTool project by pre-installing all dependencies. By removing the source code after dependency installation, we ensure that the image only contains what's necessary for future builds, thus aligning with our goal to reduce build and deployment times significantly. This change addresses part of the acceptance criteria for speeding up the deployment process by preparing and maintaining a base image with all pre-compiled dependencies required by the application.
This commit introduces a new Terraform module instantiation for creating an AWS ECR repository dedicated to the backend base Docker image. The repository, named `backend-base`, is aimed at storing pre-compiled dependencies for the backend, in line with the project's goal to speed up build and deployment times. Changes to `infra/terraform/main.tf` include the addition of the `govtool-ecr-backend-base` module, specifying the `backend-base` as the repository name. This ensures a separate and organized repository for the base image, facilitating better management and access control. Furthermore, the AWS IAM policy `cicd_ecr` is updated to include permissions for the new `backend-base` repository. This adjustment allows CI/CD pipelines to push and pull images, ensuring seamless integration with existing workflows. The outputs are also expanded to include the URL of the newly created `backend-base` ECR repository, enhancing accessibility for deployment scripts and CI/CD pipelines. This setup supports the acceptance criteria by establishing the necessary infrastructure to store and manage the base image, ultimately contributing to reduced deployment times.
This commit updates the Dockerfile for the GovTool backend to utilize the newly created base image, which contains all necessary pre-compiled Haskell dependencies. By changing the base image from `haskell:9.2-buster` to the custom `backend-base` image hosted on AWS ECR, the build process for the backend Docker image is significantly optimized. The key changes include: - Modifying the FROM directive to point to the `backend-base` image, ensuring that the environment is pre-setup with all the dependencies required for the backend build. - Simplifying the build command to `cabal build` since `cabal update`, `cabal configure`, and dependency installations are no longer necessary, being already handled in the base image creation process. This adjustment ensures that the backend Dockerfile leverages the pre-compiled dependencies in the base image, directly contributing to a faster build and deployment process. This change aligns with the project's acceptance criteria by ensuring that only the application-specific code is compiled during the build process, leveraging the efficiencies offered by the base image.
Updates the `scripts/govtool/Makefile` to require Docker login (`docker-login` target) before executing push operations for backend and frontend images, as well as before deploying the stack. This change ensures that users are authenticated with Docker, specifically with the AWS ECR, before attempting to push or pull images. This modification is critical for maintaining security and access control, particularly for actions involving sensitive operations such as pushing images to a repository. By integrating the `docker-login` prerequisite to the `push-backend`, `push-frontend`, and `deploy-stack` targets, we enforce a workflow that aligns with best practices for Docker image management and deployment. This approach safeguards against unauthorized access and ensures that the CI/CD pipeline operates smoothly with authenticated sessions. This change directly supports the project's operational security and efficiency, particularly in the context of speeding up the deployment process by leveraging pre-compiled Docker images.
This commit refines the `scripts/govtool/Makefile` to ensure all Docker commands utilize the predefined `docker` variable instead of hardcoded `docker` command references. This change enhances the flexibility and configurability of the build scripts, allowing for seamless integration with environments where Docker commands might need to be wrapped or aliased (e.g., for logging, security, or versioning purposes). By replacing direct `docker` and `docker compose` command calls with the `$(docker)` variable, we standardize command execution and increase the scripts' adaptability to different CI/CD environments. This adjustment is particularly relevant for operations like deploying the stack, destroying specific containers and volumes, and toggling maintenance mode, ensuring that all Docker interactions are executed through a single, configurable point of reference. This update supports the broader goal of improving the project's build and deployment process efficiency, aligning with best practices for script maintenance and execution flexibility.
This commit enhances the `scripts/govtool/Makefile` by introducing new targets `build-backend-base` and `push-backend-base`. These additions facilitate the process of building and pushing the base Docker image for the backend, which contains all necessary pre-compiled dependencies to expedite subsequent builds. - The `build-backend-base` target is responsible for building the base image using the `Dockerfile.base` located in the `govtool/backend` directory. It tags the image with the commit hash, providing a clear linkage between the image and the source code state from which it was built. - The `push-backend-base` target, dependent on a successful Docker login (`docker-login`), pushes the newly built base image to the configured repository. This ensures that the image is available for use by the CI/CD pipelines and developers for building the actual backend application image. These targets are crucial for implementing the strategy to speed up the backend build and deployment process by leveraging a Docker image with pre-compiled dependencies. This approach aligns with the acceptance criteria set forth for the project, aiming at reducing build and deployment times through efficient dependency management.
This commit introduces a significant enhancement to the Docker build process for the backend by making the base image versioning dependent on the `vva-be.cabal` file. This change ensures that the base Docker image is closely tied to the project's Haskell dependencies, as specified in the cabal file. - In `govtool/backend/Dockerfile`, an `ARG BASE_IMAGE_TAG` is introduced, allowing for dynamic specification of the base image tag during the build process. This facilitates using different versions of the base image as dependencies evolve. - The `scripts/govtool/Makefile` is updated to calculate the `base_backend_image_tag` by hashing the `vva-be.cabal` file. This hash becomes the tag for the base image, ensuring that any changes in the cabal file result in a new base image version. This mechanism automates the versioning process, ensuring that the backend is always built against the correct set of pre-compiled dependencies. By linking the base image version directly to the cabal file's hash, this approach guarantees that any changes in dependencies are automatically accounted for, thereby streamlining the build process and enhancing consistency across builds.
This commit optimizes the build process for the backend base Docker image in the `scripts/govtool/Makefile`. It introduces conditional logic to build or push the base image only if it doesn't already exist in the repository, significantly reducing unnecessary builds and pushes, thereby saving time and resources. Key changes include: - The `build-backend` target now depends on `build-backend-base`, ensuring the base image is available before building the backend image. However, the base image will only be built if it is not already present, as checked by `docker manifest inspect`. - Similarly, `push-backend-base` incorporates a condition to push the base image only if it is not already available in the repository, using `docker manifest inspect` to verify its existence. This logic prevents redundant builds and uploads of the base image when no changes have occurred to the `vva-be.cabal` file, making the CI/CD pipeline more efficient. By building the base image only when necessary, we align with best practices for Docker image management and optimize our deployment workflow.
placek
force-pushed
the
chore/246-prepare-a-base-image-for-backend-to-speed-up-current-process-of-building-application-for-deployment
branch
from
ff28282 to
783c582
Compare
placek
requested review from
adgud,
kickloop,
MSzalowski and
jankun4
as code owners
adgud
approved these changes
Feb 23, 2024
MSzalowski
approved these changes
Feb 23, 2024
placek
deleted the
chore/246-prepare-a-base-image-for-backend-to-speed-up-current-process-of-building-application-for-deployment
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Throughout this PR, we've implemented a series of enhancements aimed at optimizing the build and deployment process for the backend by leveraging Docker technologies:
These changes collectively contribute to a more efficient, secure, and maintainable build and deployment process, aligning with the project's goals of speeding up deployment times and facilitating quicker iterations.