Circle-CI change to deploy to ubuntu machine

[mahalakshme]

Acceptance criteria:

• Circle CI change to do deploy of Avni-server to Ubuntu machine(using avni-infra ansible scripts) using JAR, similar to the way we do for Integration-service or ETL-service
• Circle CI change to do deploy avni-webapp to Ubuntu machine using tar file(using avni-infra ansible scripts), similar to the way we do for Integration-admin-app
• Circle CI change to do deploy rules-server(using avni-infra ansible scripts) to Ubuntu machine
• Can use a test ubuntu machine(that can be later made as our staging maching) to test the same
• Migrate staging and prerelease to Ubuntu machine during development
• Make necessary changes for prod in circle-ci config but retain the old workflow in circle-ci as well, just in case. We can discuss if there exists a better option.
• Add steps for deployment to prod if any exists in readme.

How to test:

• To make sure during testing: Deployment should happen with single-click in circle-ci on server, webapp and rules-server.

Reference documentation:

https://avni.readme.io/docs/on-premise-setup-of-avni-without-cloud-services#avni-native-components-deploy

Pending:

Pre-rollout:

• Take backups of prod logs from old server if required.

Rollout:

• Start new prod server.
• Deploy latest production version of server, webapp and rule server to new_prod.
• Change 'Default' listener rule on HTTPS listener on prod-openchs-load-balancer to forward to new-prod-tg
• Stop old prod server

Rollback:

• Start old prod server
• Change 'Default' listener rule on HTTPS listener on prod-openchs-load-balancer to forward to prod-openchs-load-balancer target group.
• Stop new prod

Cleanup after successful rollout:

• stop old prod ec2 instance
• update new prod server ip into ssh.server.openchs.org and ssh.prod.avniproject.org route53 entries
• clean up .circleci/config.yml in server, webapp and rules-server to remove older deployment steps and jobs
• update infra inventories to use ssh..avniproject.org urls
• delete ssh.new.avniproject.org / openchs.org route53 entries
• delete new.avniproject.org / openchs.org route53 entries
• delete rule with priority 3 (forward to new--tg) on staging, prerelease and prod load balancers
• delete old staging, prerelease and prod target groups
• delete old staging, prerelease and prod ec2 instances.

How to test:

• Test rules-server, avni-server logs
• Configration of rules-server, avni-server, newrelic should be same as old.
• Configuration for the new AWS service/component usage same as old one
• Background job should work as before
• Media, ETL and Integration services should work correctly
• The logs in old machine should not appear
• newrelic working - can try with one time enabling or a different account
• correct versions of the app installed

[1t5j0y]

Staging (built from master/main) and Prerelease (built from 5.1.0) moved to Ubuntu servers.
No change to deployment workflow for developers.
Prod ubuntu server setup and deployed from 5.1.0. New prod avni server service stopped to prevent any conflict with prod. URL for new server: https://newapp.avniproject.org
Client APK to test against new prod server available at s3://samanvay/openchs/prod-apks/avni-x86_64--prod-5.1.1-new-ubuntu-server.apk

[himeshr]

This is moved to In-QA lane, as this is currently getting validated as part of our development and deployment activities.
No need for explicit QA.

[himeshr]

Prod Task status:

Completed Prod roll-out:

• Start new prod server.
• Deploy latest production version of server, webapp and rule server to new_prod.
• Change 'Default' listener rule on HTTPS listener on prod-openchs-load-balancer to forward to new-prod-tg
• Stop old prod server

Completed following testing :

• Test rules-server, avni-server logs
• Configration of rules-server, avni-server, newrelic should be same as old.
• Configuration for the new AWS service/component usage same as old one
• Background job should work as before
• Media, ETL and Integration services should work correctly
• The logs in old machine should not appear
• newrelic working - can try with one time enabling or a different account
• correct versions of the app installed

[himeshr]

Final Pending tasks:

Cleanup after successful rollout:

• stop old prod ec2 instance
• ENable termination protection on all required ec2 instances
• update new prod server ip into ssh.server.openchs.org and ssh.prod.avniproject.org route53 entries
• clean up .circleci/config.yml in server, webapp and rules-server to remove older deployment steps and jobs
• update infra inventories to use ssh..avniproject.org urls
• delete ssh.new.avniproject.org / openchs.org route53 entries
• delete new.avniproject.org / openchs.org route53 entries
• delete rule with priority 3 (forward to new--tg) on staging, prerelease and prod load balancers
• delete old staging, prerelease and prod target groups
• delete old staging, prerelease and prod ec2 instances.

[himeshr]

Final Pending tasks:

Cleanup after successful rollout:

• stop old prod ec2 instance
• ENable termination protection on all required ec2 instances
• update new prod server ip into ssh.server.openchs.org and ssh.prod.avniproject.org route53 entries
• clean up .circleci/config.yml in server, webapp and rules-server to remove older deployment steps and jobs
• update infra inventories to use ssh..avniproject.org urls
• delete ssh.new.avniproject.org / openchs.org route53 entries
• delete new.avniproject.org / openchs.org route53 entries
• delete rule with priority 3 (forward to new--tg) on staging, prerelease and prod load balancers
• delete old staging, prerelease and prod target groups
• delete old staging, prerelease and prod ec2 instances.

Completed all post-deploy steps