-
Notifications
You must be signed in to change notification settings - Fork 23
MR Lead Onboarding and Reference
Automatic Deployments to Dev/Production When code is pushed to MYR, MYR-admin, or MYR-backend on the dev or master branch it triggers a CI pipeline to build a docker image for that repository. The results of this image is pushed to the Quay registry as one of the following images:
These images only contain what are in their respective repositories, and are NOT COMPLETE AND WILL NOT RUN.
Once the image is pushed to Quay, a repository dispatch is then sent to the MYR-Build repository. This triggers a pipeline to run, tying all 3 intermediate images together to make a complete image, called myr on Quay.
On the dev/production servers there are 2 docker containers running (using docker-compose to orchestrate them), the MYR image and watchtower. Watchtower is used to keep the MYR image fully up to date.
All images have tags showing which branch the image was built from. The dev tag is used for dev branches and used on dev.learnmyr.org The master tag is used for master branches and used on learnmyr.org
NOTE: To deploy a new version of MYR to production, simply push the dev branch to master on GitHub. On GitHub, switch to the dev branch, click “Contribute”, then make a pull request from dev to master. Then, merge the changes and it should automatically appear on MYR.
NOTE: If I.T. reboots the server and an older version of MYR is running on learnmyr.org, there are a few things to check.
Check to make sure both docker containers are running on the production server. SSH to the server and run sudo docker ps. There should be two containers, one called app_watchtower_1, and one called app_MYR_1. They should both say “Up” for status.
If that is not the issue, run sudo su app. This makes you the user “app” then run pm2 list. There should NOT be anything in the table other than “App name”, “id”, etc. If there is, run the command pm2 cleardump, as that should get rid of any version that is left of the old system.
In order to manually deploy MYR, you first must pull from each of the repositories that have changed since the last version. In each of the repositories that have changed since the previous version, you must run the commands:
npm installnpm build
Once that is completed, change to the frontend directory and npm run prod
This will bring all the files from the different repositories together so that it can be run. The last thing to do is reload the node process. This is done by using pm2, to find the id of the process, run pm2 list. Then to restart the process, run pm2 restart <id (usually 0)>.
For MYR, Dependabot is used to help keep dependencies up to date. This happens automatically, with Pull Requests opened on the GitHub repos. If you cannot re-run CI checks on a failed PR (currently we have a flaky test), you can push an empty commit to force it. This can be done via git commit --allow-empty -m .
If there is an instance in which you only want a certain thing to be pushed to master, and not all the changes currently on dev, there are a couple ways to do it. The first way is to use git cherry-pick (https://git-scm.com/docs/git-cherry-pick) which allows you to choose one specific commit and then insert it into a different branch. This can be riskier, as you are doing this in the original MYR repo rather than your own fork. A second, safer way is to do this directly in GitHub. This is more practical if you are only changing one file. On the master branch, you can select a file, then edit that file to contain the changes you want. Then, create a pull request of these changes, make sure the checks pass, then merge into master. NOTE: As of the time of writing this (John K), I have only done the second method, as I was not able to make the cherry-pick command work, but it is worth looking into more, as it is more useful for larger commits.
The SSL certificates used to encrypt HTTP traffic to and from the MYR servers expire once a year. Currently this is going to happen on September 27th, 2023. In order to renew, contact Fred to open a ticket with IT to renew it.
Steps:
- MUST have SSH access to the servers. Have Fred or the current Lead add you. Also, have them make you a sudoer.
- Obtain a certificate renewal from IT. Either they will preemptively do it, or you will have to ask Fred to contact them about it. Once you get the email, download the file under “as Certificate only, PEM encoded”
- Once you SSH into the MYR server, cd into /etc/ssl. Now, you will create a new .crt file, and you can do this by running the line “sudo nano .crt”. This will open a window to write a new file.
- Open the original PEM file you downloaded earlier, and you can do this in VS Code or another IDE. Copy the certificate into the .crt file you have open in the terminal. It should start with something like “------BEGIN CERTIFICATE-------” and end with something like “-------END CERTIFICATE-------”. Now save this file.
- Now, this new .crt file should be in the /etc/ssl directory. Now you should cd into /etc/nginx/sites-enabled. You will run the line “sudo nano node.conf”. This opens a configuration file for you to change.
- The ONLY thing you change in this file is the line where it says “ssl_certificate”, and you will replace the path for the old certificate with the path for the new one. So, right now it says “/etc/ssl/learnmyr_org_cert.crt”, and you will change it to “/etc/ssl/.crt”. Save this file.
- Finally, run the command “sudo service nginx restart”. Now you’re all set!
As of right now (December 6, 2022), production and dev both have 5 security headers in place. To check the security headers, you can use a tool called shcheck. The instructions to use that tool are found online. If you want to edit/add security headers, go to either server, and cd into /etc/nginx/sites-enabled. The file node.conf has these headers. Simply run sudo nano node.conf, and this allows you to edit the file. Once done editing the file, save it and exit, then run sudo service nginx restart.
As of now, the header for Content Security Policy does NOT work and if added, will crash the website. If you are playing around with headers and the website crashes, just delete the headers you added and restart with the sudo service nginx restart command. If that does not work, run sudo reboot, which will completely reboot the server.
This page can be helpful: https://webdock.io/en/docs/how-guides/security-guides/how-to-configure-security-headers-in-nginx-and-apache
ssh into the dev server (better to do this on dev as we don’t want prod to break) Run the command “mongodump”. This should make a copy of the database and it will be stored in your vm as a directory called “dump” Exit ssh, then sftp into the dev server. This is the same exact command as ssh, but you just type sftp instead of ssh Type the command “get -r dump”. This command will copy the files in the dump directory into your local machine. Helpful info for sftp: SFTP Documentation
It is very important that the developers have this key in BOTH the front-end and back-end code, in the .env file. If it is only in one, then the developer will encounter an error every time the code is run, as well as trying to run actions that interact with the back-end. Overall, make sure all the developers implement this step.
Sometimes when changing, either upgrading or downgrading, packages, urls are changed to have git+ssh instead of https at the start. This will cause the ci checks to fail, so make sure to change those back to https