Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple the process of image build from the current flow #211

Closed
IlanZuckerman opened this issue Feb 21, 2021 · 10 comments · Fixed by #221 · May be fixed by #223
Closed

Decouple the process of image build from the current flow #211

IlanZuckerman opened this issue Feb 21, 2021 · 10 comments · Fixed by #221 · May be fixed by #223
Assignees

Comments

@IlanZuckerman
Copy link
Collaborator

Currently the docker image is being built and deployed on the same instance, and in the same flow.
This needs to be decoupled onto different stages.
The image needs to be pushed to quay.io

@lmilbaum
Copy link
Contributor

Acceptance Criteria is missing

@IlanZuckerman
Copy link
Collaborator Author

@lioramilbaum I couldnt find on web any solution on how to decouple the image build from current flow (Vagrantfile) without going backwards again when we used the external shell script.
In my understanding, if we wish to decouple the image build (and pushing it to remote repo) from current flow, we need to have completely independent flow (or maybe even project) which will handle the image build + push to remote. In that case, the Vagrant flow and github actions flow will have to pull the image from remote source, and run it as a container.
In that case we would have to maintain 3 different flows, and be able to connect them between each other. This might be a serious overhead.

Pls correct me if I'm wrong.
There is another question, what exactly do we wish to improve here? The overall time that it takes for the complete flow to complete?

@lmilbaum
Copy link
Contributor

To answer your question, what we want to achieve by decoupling the image build from the deployment process:
The image which is used in prod environment should be built from a specific set of sources. The current flow where we are able to deploy to prod from our fork repo is not the desired state.
I have googled a bit and found the following article:
https://www.dataversity.net/what-a-lovely-decouple-why-decoupling-deploy-from-release-is-a-game-changer

I am sure that if you google it, you might find additional ones.

@IlanZuckerman
Copy link
Collaborator Author

IlanZuckerman commented Apr 11, 2021

'beyond' image is now hosted on our namespace 'quay.io/redhat-beyond/beyond'
To pull it:
docker pull quay.io/redhat-beyond/beyond

$ docker pull quay.io/redhat-beyond/beyond Using default tag: latest latest: Pulling from redhat-beyond/beyond 6e0aa5e7af40: Pull complete d47239a868b3: Pull complete 49cbb10cca85: Pull complete d5695fbb345b: Pull complete 41cf14d8ca2d: Pull complete c55777d9cdd4: Pull complete 3ae64b5d8ceb: Pull complete a09a5891304b: Pull complete Digest: sha256:f1b196c017e22347812a5f30d1b49b1b1128f9c1e6ebf6895879f8ae4f730f64 Status: Downloaded newer image for quay.io/redhat-beyond/beyond:latest quay.io/redhat-beyond/beyond:latest ilan at localhost in ~/git/baboon on master*

$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
quay.io/redhat-beyond/beyond latest 3856b16c991e 6 minutes ago 493MB

@IlanZuckerman
Copy link
Collaborator Author

IlanZuckerman commented Apr 12, 2021

@lioramilbaum as a part of this effort, I now need to make the following possible (correct me if I;m wrong):

  • AWS EC2 service needs to be granted a permission to pull our image from 'quay' (currently it is failing [1]).

In order to make this happen, I created a robot account in our namespace on quay. But I cant find any info about how should i proceed from here.
Can you pls point me?

[1]:
_$ vagrant up stage --provider=aws
Bringing machine 'stage' up with 'aws' provider...
==> stage: Warning! The AWS provider doesn't support any of the Vagrant
==> stage: high-level network configurations (config.vm.network). They
==> stage: will be silently ignored.
==> stage: Warning! You're launching this instance into a VPC without an
==> stage: elastic IP. Please verify you're properly connected to a VPN so
==> stage: you can access this machine, otherwise Vagrant will not be able
==> stage: to SSH into it.
==> stage: Launching an instance with the following settings...
==> stage: -- Type: t2.micro
==> stage: -- AMI: ami-0701e7be9b2a77600
==> stage: -- Region: eu-west-1
==> stage: -- Keypair: aws_key_pair
==> stage: -- Subnet ID: subnet-1c86b754
==> stage: -- Security Groups: ["sg-71952b3f"]
==> stage: -- Block Device Mapping: []
==> stage: -- Terminate On Shutdown: false
==> stage: -- Monitoring: false
==> stage: -- EBS optimized: false
==> stage: -- Source Destination check:
==> stage: -- Assigning a public IP address in a VPC: true
==> stage: -- VPC tenancy specification: default
==> stage: Waiting for instance to become "ready"...
==> stage: Waiting for SSH to become available...
==> stage: Machine is booted and ready for use!
==> stage: Rsyncing folder: /home/ilan/git/baboon/ => /vagrant
==> stage: - Exclude: [".vagrant/", "venv/", ".git/", ".idea/"]
==> stage: Running provisioner: docker...
stage: Installing Docker onto machine...
==> stage: Running provisioner: docker_compose...
stage: Checking for Docker Compose installation...
stage: Getting machine and kernel name from guest machine...
stage: Downloading Docker Compose 1.28.2 for Linux x86_64
stage: Downloaded Docker Compose 1.28.2 has SHA256 signature e984402b96bb923319f8ac95c8d26944e12d15692ab0fea4fff63ddfd5fd8d64
stage: Uploading Docker Compose 1.28.2 to guest machine...
stage: Installing Docker Compose 1.28.2 in guest machine...
stage: Symlinking Docker Compose 1.28.2 in guest machine...
stage: Running docker-compose rm...
stage: No stopped containers
stage: Running docker-compose build...
==> stage: beyond uses an image, skipping
==> stage: db uses an image, skipping
==> stage: Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
stage: Running docker-compose up...
==> stage: Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
==> stage: Creating network "vagrant_default" with the default driver
==> stage: Pulling beyond (beyond:)...
==> stage: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.
==> stage:
stage: Continue with the new image? [yN]
==> stage: pull access denied for beyond, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

/usr/local/bin/docker-compose-1.28.2 -f "/vagrant/docker-compose.yml" up -d

Stdout from the command:

Continue with the new image? [yN]

Stderr from the command:

Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating network "vagrant_default" with the default driver
Pulling beyond (beyond:)...
The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.

pull access denied for beyond, repository does not exist or may require 'docker login': denied: requested access to the resource is denied_

@lmilbaum
Copy link
Contributor

Have you tried to perform manually what you expect the automation to do?

@IlanZuckerman
Copy link
Collaborator Author

Yes, locally (manually) it works fine. I am able to pull the image from quay and continue with the flow.
No special permissions are needed there.

@lmilbaum
Copy link
Contributor

What about spinning the environment with Vagrant, ssh-ing and performing the steps there?

@IlanZuckerman
Copy link
Collaborator Author

Thanks, i was managed to resolve. the reason was that i used not complete image name. The issue is now resolved, and patch waiting for review.

What I had (and did not work):
FROM beyond

What should be:
FROM quay.io/redhat-beyond/beyond

@lmilbaum
Copy link
Contributor

Thanks, i was managed to resolve. the reason was that i used not complete image name. The issue is now resolved, and patch waiting for review.

What I had (and did not work):
FROM beyond

What should be:
FROM quay.io/redhat-beyond/beyond

Cool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants