diff --git a/README.md b/README.md index 02edd4f22..6c6056991 100644 --- a/README.md +++ b/README.md @@ -1,47 +1,32 @@ # Get Help Buying for Schools -[![Maintainability][codeclimate-badge]][codeclimate-report] -![CI][ci-badge] -![Deploy][deploy-badge] +[Github Repo](https://github.com/DFE-Digital/buy-for-your-school) +| [Github Packages](https://github.com/dfe-digital/buy-for-your-school/pkgs/container/buy-for-your-school) + +![main](https://github.com/DFE-Digital/buy-for-your-school/actions/workflows/ci-full-pipeline.yml/badge.svg?branch=main) A service to help school buying professionals create tender documents that comply with the relevant government policy. These tender documents can then be used to start a procurement process saving schools time and money. -## Reusable Code - -This project uses **[DfE Sign-In][dsi]** for authentication. -If your department project does the same, the code in `./lib/dsi` could help you. - -## Documentation - -Run `$ yardoc` to generate documentation for the project in `/documentation` and then `$ open ./documentation/index.html` to open in the browser. - -## Changelog +|Environment|URL| +|-|-| +|Production|https://www.get-help-buying-for-schools.service.gov.uk| +|Staging|https://staging.get-help-buying-for-schools.service.gov.uk| +|Development|https://dev.get-help-buying-for-schools.service.gov.uk| -When making a change, update the [changelog](CHANGELOG.md) using the -[Keep a Changelog 1.0.0][keep-a-changelog] format. +## Getting Started with Development -## Architecture Decision Records +Please read the [Getting Started](doc/getting-started.md) documentation for help on setting up your local environment and getting access to required systems. -We use ADRs to document architectural decisions managed with [adr-tools][adr]. - -## Access - -| Environment | URL | -| :---------- | :-----------------------------------------------------------: | -| Development | http://localhost:3000 | -| Research | https://buy-for-your-school-research.london.cloudapps.digital | -| Staging | https://staging-get-help-buying-for-schools.education.gov.uk | -| Production | https://get-help-buying-for-schools.education.gov.uk | - - - ---- +## Documentation -[adr]: https://github.com/npryce/adr-tools -[ci-badge]: https://github.com/DFE-Digital/buy-for-your-school/actions/workflows/continuous-integration.yml/badge.svg -[codeclimate-badge]: https://api.codeclimate.com/v1/badges/f119cce1678a8a67cca7/maintainability -[codeclimate-report]: https://codeclimate.com/github/DFE-Digital/buy-for-your-school/maintainability -[deploy-badge]: https://github.com/DFE-Digital/buy-for-your-school/actions/workflows/deploy.yml/badge.svg -[dsi]: https://services.signin.education.gov.uk/ -[keep-a-changelog]: https://keepachangelog.com/en/1.0.0/ +- [Accessibility](doc/accessibility.md) +- [Azure infrastructure](doc/azure.md) +- [Console Access](doc/console-access.md) +- [Contentful](doc/contentful-updates.md) +- [Contentful Webhooks](doc/webhooks.md) +- [Continuous Integration](doc/continuous-integration.md) +- [Logging](doc/logging.md) +- [Managing Environment Variables](doc/managing-environment-variables.md) +- [Notify](doc/notify.md) +- [Roles and Portals](roles-and-portals.md) diff --git a/doc/azure.md b/doc/azure.md new file mode 100644 index 000000000..4eafe6d6b --- /dev/null +++ b/doc/azure.md @@ -0,0 +1,10 @@ +# Azure + +## Resource groups + +|Environment|Azure Resource Group|URL| +|-|-|-| +|Production|s174p01-ghbs-app|https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/1871efd3-900d-4908-bf66-1b268ab28f10/resourceGroups/s174p01-ghbs-app| +|Staging|s174t01-ghbs-app|https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/cb3dd736-2dc8-4296-9894-16d3ecabe85d/resourceGroups/s174d01-ghbs-app| +|Development|s174d01-ghbs-app|https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/cb3dd736-2dc8-4296-9894-16d3ecabe85d/resourceGroups/s174d01-ghbs-app| + diff --git a/doc/cloud-foundry-utilities.md b/doc/cloud-foundry-utilities.md deleted file mode 100644 index 44d049e81..000000000 --- a/doc/cloud-foundry-utilities.md +++ /dev/null @@ -1,33 +0,0 @@ -# Cloud Foundry Utilities - -## Prerequisites - -You must already have Cloud Foundry installed, user account setup and be able to access the [console](file.console-access.html). - -## Exporting Data to CSV - -1. From a local terminal login to Cloud Foundry and SSH into the specified application. - ``` - $ cf ssh REDACTED-APP-NAME-PROD - ``` -1. Enter app directory - ``` - $ cd /srv/app/ - ``` -1. Access rails console and paste your script (as per example provided in step 2, modified) - ``` - $ PATH=$PATH:/usr/local/bundle/bin:/usr/local/bin rails console - ``` -1. Example code to dump to CSV file. - - ```ruby - require 'csv' - file = "#{Rails.root}/public/example_data.csv" - records = YOUR-MODEL.order(:created_at) - csv_headers = ["id", "attr_1", "attr_2"] - CSV.open(file, 'w', write_headers: true, headers: csv_headers) do |writer| - records.each do |record| - writer << [record.id, record.attr_1, record_attr_2] - end - end - ``` \ No newline at end of file diff --git a/doc/console-access.md b/doc/console-access.md index 4fdcaf1bc..3168b9935 100644 --- a/doc/console-access.md +++ b/doc/console-access.md @@ -1,51 +1,13 @@ # Console access -We may need a way to access live environments for debugging or incident management purposes in future. +NOTE: you will need Azure access to do this -If we do need to open a rails console on production we should pair through the commands we execute to mitigate the risk of data loss. +Please read [this documentation on console access](https://learn.microsoft.com/en-us/azure/container-apps/container-console?tabs=bash#azure-portal) for more information on connecting to a container app console. -## Prerequisites +## Console access links -You must have an account that has been invited to the Government Platform as a Service (GPaaS) account. DfE PaaS organisation administrators should be able to invite you if you [request in DfE's #digital-tools-support Slack channel](https://ukgovernmentdfe.slack.com/archives/CMS9V0JQL). - -You must have have been given 'Space developer' access to the intended space, for example "sct-prod". Note 'Space manager' is a separate role and does not include all `Space developer` permissions. - -[You can sign in to check your account and permissions here](https://admin.london.cloud.service.gov.uk). - -## Access - -1. From a local terminal login to Cloud Foundry and select the intended space - ``` - $ cf login -a api.london.cloud.service.gov.uk -u REDACTED@digital.education.gov.uk - ``` -1. See all available spaces - ``` - $ cf spaces - ``` -1. Change space - ``` - $ cf target -s - ``` -1. View available services - ``` - $ cf apps - ``` -1. Connect to the environment (in this case production) - ``` - $ cf ssh - ``` -1. Navigate to the application - ``` - $ cd /srv/app - ``` -1. Run the intended commands - ``` - $ export PATH="$PATH:/usr/local/bin" - $ /usr/local/bin/ruby bin/rails c - ``` - - or - - ``` - $ /usr/local/bin/ruby bin/rake db:seed - ``` +|Enviroment|Console Link| +|--|--| +|Development|[Console](https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/cb3dd736-2dc8-4296-9894-16d3ecabe85d/resourceGroups/s174d01-ghbs-app/providers/Microsoft.App/containerApps/devghbs-buyforyourschool/console)| +|Staging|[Console](https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/bc55c4cf-d75d-42d1-9017-a457b6b2cfc8/resourceGroups/s174t01-ghbs-app/providers/Microsoft.App/containerApps/stagghbs-buyforyourschool/console)| +|Production|[Console](https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/1871efd3-900d-4908-bf66-1b268ab28f10/resourceGroups/s174p01-ghbs-app/providers/Microsoft.App/containerApps/prodghbs-buyforyourschool/console)| diff --git a/doc/continuous-integration.md b/doc/continuous-integration.md new file mode 100644 index 000000000..accb1a0e3 --- /dev/null +++ b/doc/continuous-integration.md @@ -0,0 +1,17 @@ +# Continuous Integration + +## Github Actions + +Automated deployments are handled by [github actions](https://github.com/DFE-Digital/buy-for-your-school/actions). + +### Deployment Secrets + +Select secrets are stored in github "Environments". We store as little secrets as possible within github and prefer to rely on permissions within Azure. + +|Secret|Description| +|-|-| +|`AZURE_SP_CREDENTIALS`|Service principle credentails stored in JSON form| +|`CONTAINER_APP_NAME`|Name of the container-app| +|`RESOURCE_GROUP_NAME`|Name of the resource group| + +For more on service principal credentials, see [here](https://learn.microsoft.com/en-us/azure/developer/github/connect-from-azure?tabs=azure-portal%2Clinux#create-a-service-principal) and [here](https://github.com/marketplace/actions/azure-login#configure-a-service-principal-with-a-secret). diff --git a/doc/dfe-sign-in.md b/doc/dfe-sign-in.md index d155b3df3..a999ebd5c 100644 --- a/doc/dfe-sign-in.md +++ b/doc/dfe-sign-in.md @@ -2,111 +2,29 @@ **_DfE Sign-In_** `DSI` is the **_Single Sign-On_** `SSO` provider for this service. -There are [testing][test-register], [pre-production][pre-prod-register] and [production][prod-register] environments for DSI that each require registration. - -## Registration - -Anyone can register for a DSI profile in each environment. -An *approver* at a supported establishment can invite users to their organisation. -[This school][test-school] has been used for testing access for local development. -Contact DfE [#digital-tools-support][digi-tools] and [#dfe_sign-in][dfe_sign-in] Slack channels for further assistance. - -## Access - -The DSI API returns a user's affiliated organisations with their type name and number. -Will restrict access to users from certain types of establishment. - -**Accept** - - 1 Community School - 2 Voluntary Aided School - 3 Voluntary Controlled School - 5 Foundation School - 6 City Technology College - 7 Community Special School - 12 Foundation Special School - 14 Pupil Referral Unit - 28 Academy Sponsor Led - 33 Academy Special Sponsor Led - 34 Academy Converter - 35 Free Schools - 36 Free Schools Special - 38 Free Schools - Alternative Provision - 40 University Technical College - 41 Studio Schools - 42 Academy Alternative Provision Converter - 43 Academy Alternative Provision Sponsor Led - 44 Academy Special Converter - -**Reject** - - 8 Non-Maintained Special School - 10 Other Independent Special School - 11 Other Independent School - 15 LA Nursery School - 18 Further Education - 24 Secure Units - 25 Offshore Schools - 26 Service Children's Education - 27 Miscellaneous - 29 Higher Education Institution - 30 Welsh Establishment - 31 Sixth Form Centres - 32 Special Post 16 Institution - 37 British Overseas Schools - 39 Free Schools - 16-19 - 45 Academy 16-19 Converter - 46 Academy 16-19 Sponsor Led - 47 Children's Centre - 48 Children's Centre Linked Site - 56 Institution funded by other government department - - ## Environments -This service has deployment environments and each is paired with a corresponding DSI environment. - -| | Enabled | DSI Env | DSI service management | -| :------------- | :----------- | :------------------- | :------------------------- | -| Development | optional | [test][test] | [manage][test-manage] | -| Staging | true | [pre-prod][pre-prod] | [manage][pre-prod-manage] | -| Research | true | [pre-prod][pre-prod] | [manage][pre-prod-manage] | -| Production | true | [prod][prod] | [manage][prod-manage] | - -DSI DNS Aliases: - -- `https://interactions.signin.education.gov.uk` -> `https://services.signin.education.gov.uk` -- `https://pp-interactions.signin.education.gov.uk` -> `https://pp-services.signin.education.gov.uk` -- `https://test-interactions.signin.education.gov.uk` -> `https://test-services.signin.education.gov.uk` - -## Development - -In development, setting the environment variable `DFE_SIGN_IN_ENABLED` to false will bypass DSI. -You can provide _any_ value in the `UID` field to sign in. -The application has matured and now requires user data provided by DSI therefore bypassing has limited use. - -Communicating with DSI in development requires a secure connection. - -Create a self-signed certificate: +There are three different DfE sign in environments, an account on one environment will not be shared across to the other environments. However, both staging and development environments both use the same DfE sign in environment so the account can be shared in effect. -`$ openssl req -x509 -sha256 -nodes -newkey rsa:2048 -days 365 -keyout localhost.key -out localhost.crt` +Below is a mapping of which applications are backed by which DfE sign in environments: -Firefox will permit the use of this certificate. On OSX, Chrome will require that certificate be trusted. -In **Keychain Access** add it to the **Certificates** in the **System Keychain**. -Use `File > Import Items` and import `localhost.crt`. -Once imported change the trust level to `Always Trust`. +| App Environment |DSI Env | DSI service management | Request Organisations | +| :------------- |:------------------- | :------------------------- | --------------------- | +| Local Dev |[test][test] | [manage][test-manage] | [test][test-request-org] | +| Development |[pre-prod][pre-prod] | [manage][pre-prod-manage] | [pre-prod][pre-prod-request-org] | +| Staging |[pre-prod][pre-prod] | [manage][pre-prod-manage] | [pre-prod][pre-prod-request-org] | +| Production |[prod][prod] | [manage][prod-manage] | [prod][prod-request-org] | +## Getting an account +### Sign up -See `Procfile.dev` for starting puma with SSL. +DfE Sign in is "self service" so in order to get an account you need to sign up yourself. Please click on each link in the "DSI Env" in the table above and follow the steps to create an account. +### Request access -## Organisations +Now you have an account you will be able to log in, but the application does further authorization with your account details. -The service leverages a user's affiliation to an organisation within DSI to control access. -The development team should use the `test` DSI environment variables in `.env.development.local`. -Developers in the test environment can [*approve* or *invite*][test-users] new team members. -The organisation **"DfE Commercial Procurement Operations"** is required to gain access beyond `/support` in each environment. -The lead developer or product manager will be approvers and can invite new team members thereby granting access to live environments. +In order to gain access to parts of the system you need to request access to the "DfE Commercial Procurement Operations" organisation. Use the links in the "Request Organisations" column of the table above, search for "DfE Commercial Procurement Operations" organisation and then request access. --- @@ -114,16 +32,19 @@ The lead developer or product manager will be approvers and can invite new team [pre-prod-register]: https://pp-profile.signin.education.gov.uk/register [pre-prod-manage]: https://pp-manage.signin.education.gov.uk/services/00487750-C9B8-414C-8746-1076885456E0/service-configuration [pre-prod-api]: https://pp-api.signin.education.gov.uk +[pre-prod-request-org]: https://pp-services.signin.education.gov.uk/request-organisation/search [prod]: https://services.signin.education.gov.uk [prod-register]: https://profile.signin.education.gov.uk/register [prod-manage]: https://manage.signin.education.gov.uk/services/9D1B3879-3495-4D3F-AB7A-ED9B8E968EFF/service-configuration [prod-api]: https://api.signin.education.gov.uk +[prod-request-org]: https://services.signin.education.gov.uk/request-organisation/search [test]: https://test-services.signin.education.gov.uk [test-register]: https://test-profile.signin.education.gov.uk/register [test-manage]: https://test-manage.signin.education.gov.uk/services/FD39DCFC-9B60-46C4-ACDC-699A2468B46F/service-configuration [test-api]: https://test-api.signin.education.gov.uk [test-users]: https://test-services.signin.education.gov.uk/approvals/users [test-school]: https://test-services.signin.education.gov.uk/approvals/50F4A834-9314-4A66-969E-C86D03821C26/users +[test-request-org]: https://test-services.signin.education.gov.uk/request-organisation/search [digi-tools]: https://ukgovernmentdfe.slack.com/archives/CMS9V0JQL [dfe_sign-in]: https://ukgovernmentdfe.slack.com/archives/C5S500XB6 diff --git a/doc/docker-useful-commands.md b/doc/docker-useful-commands.md deleted file mode 100644 index 943106928..000000000 --- a/doc/docker-useful-commands.md +++ /dev/null @@ -1,14 +0,0 @@ -# Docker - useful commands - -## General - -- `$ docker ps` shows which containers are running -- `$ docker-compose up redis` starts up redis (or substitute another service) in isolation to test it's working - -## Reset - -- `$ docker-compose down --remove-orphans` -- `$ docker rm -f $(docker ps -a -q)` -- `$ docker volume rm $(docker volume ls -q)` -- Confirm no containers are running with `$ docker ps` -- Restart the container image with `$ script/server` diff --git a/doc/getting-started.md b/doc/getting-started.md index 9c4e96f6f..38d1e4780 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -1,165 +1,63 @@ # Getting started -Using [Docker](https://docs.docker.com/docker-for-mac/install) has high parity, -you don't have to install any dependencies to run the app, as the project is run -in an isolated container, but it takes longer to run. +Take the following steps to getting started: -The preferred option is to work with Docker (option 1). +- [Setup your development environment](getting-started/development-environment.md) +- [Getting your DfE Sign In Accounts](dfe-sign-in.md) +- [Getting commit access to the repo](#getting-commit-access) +- [Starting the application](#starting-the-application) +- [Grant yourself roles](roles-and-portals.md) +- [Seed your local environment](#seed-your-local-environment) -## Environment Variables -- Obtain environment variable secrets from another member of the development team -- Copy `/.env.example` to `/.env.development.local` and populate +## Getting Commit Access -## Utility Scripts +Ask a developer with github access to add your github handle as a "Collaborator", this will allow you to create pull requests. -Within `/scripts`, there are a number of convenience scripts for running processes in containers. +In order to deploy to the `development` environment you need to be allowed to push to the `development` branch, so ask for your github handle to be added within the branch protections. ---- +## Starting the application -## 1. Using Docker - -`$ script/build` will build and tag the container images used in docker compose. - -### Development - -`$ script/server` will launch the rails development server - - NB: run script/build if the image ghbs:dev does not exist locally - ----------------------------------------------------------------- - [+] Running 8/8 - ⠿ Network ghbs_default - ⠿ Network ghbs_ghbs - ⠿ Volume "ghbs_db_dev" - ⠿ Volume "ghbs_cache_dev" - ⠿ Container ghbs_db - ⠿ Container ghbs_cache - ⠿ Container ghbs_worker - ⠿ Container ghbs_dev - -The project uses [Pry](https://github.com/pry/pry) with [Byebug](https://github.com/deivid-rodriguez/byebug) -in place of [IRB](https://guides.rubyonrails.org/command_line.html#bin-rails-console) - -`$ script/console` will enter a rails console - -### Test - -`$ script/spec` will run the whole test suite, but can accept an optional spec path, which it will output in documentation format. - - NB: run script/build if the image ghbs:test does not exist locally - ----------------------------------------------------------------- - [+] Running 6/6 - ⠿ Network ghbs_ghbs Created - ⠿ Volume "ghbs_cache_test" Created - ⠿ Volume "ghbs_db_test" Created - ⠿ Container ghbs_db Created - ⠿ Container ghbs_cache Created - ⠿ Container ghbs_chrome Created - [+] Running 3/3 - ⠿ Container ghbs_cache Started - ⠿ Container ghbs_chrome Started - ⠿ Container ghbs_db Started - -### CICD - -Please note that in the pipeline the script `script/test` is run, which is responsible -for chaining together dependency updates, migrations, testing, linting and security checks. - ---- - -## 2. Without Docker (local installation) - -### Dependencies - -**OSX** - -- Install [Homebrew](https://brew.sh) -- Copy `/Brewfile.example` to `/Brewfile` and uncomment any required dependencies like `pandoc` and `basictex` -- Run `$ brew bundle` to install any missing dependencies - -### Development - -- The assets need to be pre-compiled by running `$ rake webpack:compile` -- Run the server using `$ bundle exec rails server` if you are bypassing DfE Sign In, -otherwise [click here](dfe-sign-in.md) for more information. -- Start the console `$ bundle exec rails console` - -### Test - -- Run test suite `$ bundle exec rspec` or `bundle exec rake spec` -- Run lint check `$ bundle exec rubocop` or `bundle exec rake rubocop` -- Run test suite and lint check `bundle exec rake` - -### Services - -ASDF can also be used to manage multiple runtime versions. Example step-by-step guide using [ASDF](https://asdf-vm.com) for dependencies. - -Postgres +To start up the application with all processes (See Procfile.dev): ``` -$ asdf plugin add postgres -$ POSTGRES_EXTRA_CONFIGURE_OPTIONS=--with-uuid=e2fs asdf install postgres latest -$ pg_ctl start -$ createuser postgres --super -$ createdb postgres +$ ./bin/dev ``` -Redis +To start up the application only (no sidekiq or asset compilation): ``` -$ asdf plugin add redis -$ asdf install redis latest -$ redis-server +$ bundle exec rails server -b 'ssl://0.0.0.0:3000?key=localhost.key&cert=localhost.crt' ``` -Node +### Manually starting asset watching and compilation -``` -$ asdf plugin add nodejs -$ asdf install nodejs latest -``` +NOTE: `./bin/dev` will do this for you automatically. -[Ruby](https://gds-way.cloudapps.digital/manuals/programming-languages/ruby.html#conventional-tooling) - alternative package managers like [Rbenv](https://github.com/rbenv/rbenv), [RVM](https://github.com/rvm/rvm), [Chruby](https://github.com/postmodern/chruby) can also be used +To watch and compile javascript: ``` -$ asdf plugin add ruby -$ asdf install ruby +$ yarn build --watch ``` -Rubygems +To watch and compile css: ``` -$ gem install bundle -$ bundle +$ yarn build:css --watch ``` -Additional install configuration (if required) - -``` -$ gem install pg -- --with-pg-config=$(asdf which pg_config) -``` - -Prepare the databases - -``` -$ rake db:prepare -$ RAILS_ENV=test rake db:prepare -``` - -## Annotations - -`rails notes` are used to provide WIP information for developers. - -## Security +## Seed your local environment -Run [Brakeman](https://brakemanscanner.org/) to highlight any security vulnerabilities: +Run the following seeds: ``` -$ brakeman +$ rails self_serve:populate_categories +$ rails case_management:seed +$ rails request_for_help:seed ``` -To pipe the results to a file: +If you have valid MS Graph credentials you can also seed the local emails: ``` -$ brakeman -o report.text +$ rails case_management:seed_shared_inbox_emails ``` diff --git a/doc/getting-started/development-environment.md b/doc/getting-started/development-environment.md new file mode 100644 index 000000000..82de2eceb --- /dev/null +++ b/doc/getting-started/development-environment.md @@ -0,0 +1,70 @@ +# Setting up your development environment + +- [Install Dependencies](#install-dependencies) +- [Setup environment variables](#setup-environment-variables) + +## Install Dependencies + +The following instructions are written with Mac OS users in mind, please seek alternative documentation for Linux or Windows users. + +### Install Ruby + +Follow the instructions here: https://rvm.io/ + +Or alternativly use the [asdf-ruby](https://github.com/asdf-vm/asdf-ruby) plugin for [asdf](https://github.com/asdf-vm/asdf) + +### Install Postgres + +``` +$ brew install postgresql@11 +``` + +You might need to start postgres running, use the following command whenever your postgres server is not running: + +``` +$ brew services restart postgresql@11 +``` + +### Install Redis + +``` +$ brew install redis +``` + +You might need to start redis running, use the following command whenever your redis server is not running: + +``` +$ brew services restart redis +``` + +### Install Chromedriver + +NOTE: you will need actual Google Chrome installed also. + +``` +$ brew install chromedriver +``` + +Ensure chromedriver is not blocked by the system when running: + +``` +$ xattr -d com.apple.quarantine `which chromedriver` +``` + +## Setup Environment Variables + +You require the `.env.development.local` file for your development environment to run, please ask an existing member of the team for an up to date copy. + +## Create a self signed SSL certificate + +``` +$ openssl req -x509 -sha256 -nodes -newkey rsa:2048 -days 365 -keyout localhost.key -out localhost.crt +``` + +## Project Setup + +``` +$ bundle install +$ bundle db:reset +$ bundle exec rails assets:precompile +``` diff --git a/doc/logging.md b/doc/logging.md index 260703cec..b2d855384 100644 --- a/doc/logging.md +++ b/doc/logging.md @@ -1,52 +1,13 @@ # Logging -## Logit +NOTE: you will need azure log access to do this -Logs for all live environments are aggregated in the [Logit serivce](https://dashboard.logit.io). +## Azure Logs -An existing developer should be able to invite you. +Read more about Log Analytics [here](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-overview) -Environment variables are used to tell the infrastructure which log stream URL to send logs to. These can be changed in GitHub Secrets within the repository: - -``` -PROD_TF_VAR_SYSLOG_DRAIN_URL= -STAGING_TF_VAR_SYSLOG_DRAIN_URL= -RESEARCH_TF_VAR_SYSLOG_DRAIN_URL= -``` - -For help putting together the drain URL, there is a [service-specific guide in -the CloudFoundry -documentation](https://docs.cloudfoundry.org/devguide/services/log-management-thirdparty-svc.html) -which includes LogIt. We use the TCP-SSL port with the `syslog-tls://` URL -scheme. - -Remember to deploy the application again to propagate environment variable changes. - -## GPaaS - -If Logit isn't working you can ask GPaaS to view the logs for individual apps using the Cloud Foundry CLI. - -Log in: - -``` -$ cf login -a api.london.cloud.service.gov.uk -u REDACTED@digital.education.gov.uk -``` - -Select the intended space: - -``` -$ cf spaces -$ cf target -s sct-staging -``` - -Find the app name for the process you want to see logs for: - -``` -$ cf apps -``` - -Ask for the logs: - -``` -$ cf logs buy-for-your-school-research -``` +|Environment|Log Link| +|-|-| +|Development|[Log](https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/1871efd3-900d-4908-bf66-1b268ab28f10/resourceGroups/s174p01-ghbs-app/providers/Microsoft.OperationalInsights/workspaces/prodghbscontainerapp/logs)| +|Staging|[Log](https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/bc55c4cf-d75d-42d1-9017-a457b6b2cfc8/resourceGroups/s174t01-ghbs-app/providers/Microsoft.OperationalInsights/workspaces/stagghbscontainerapp/logs)| +|Production|[Log](https://portal.azure.com/#@platform.education.gov.uk/resource/subscriptions/cb3dd736-2dc8-4296-9894-16d3ecabe85d/resourceGroups/s174d01-ghbs-app/providers/Microsoft.OperationalInsights/workspaces/devghbscontainerapp/logs)| diff --git a/doc/managing-environment-variables.md b/doc/managing-environment-variables.md index 41ddef3ae..bf1d1eb16 100644 --- a/doc/managing-environment-variables.md +++ b/doc/managing-environment-variables.md @@ -18,114 +18,6 @@ If the environment variable is critical whereby it is required to start the appl ## Live environments -We use [GitHub Secrets as the canonical source of our environment variables](https://github.com/DFE-Digital/buy-for-your-school/settings/secrets/actions). +Currently in order to add a new environment variable, they will need to be added to the terraform .tfvars file for that particular environment and a manual deployment must take place. -Environment variables are all managed in the same way. Secrets are kept secret by taking advantage of the protection GitHub Secrets offers. Configuration options which don't necessarily need to be kept secret are still managed through environment variables as part of being [a 12 factor app](https://12factor.net/config). - -### Adding a variable - -#### Application - -A variable will usually be required to be added into each environment through [a new repository secret](https://github.com/DFE-Digital/buy-for-your-school/settings/secrets/actions/new). For this we use prefixes in our environment variable names: - -``` -APP_ENV_PROD_ -APP_ENV_STAGING_ -APP_ENV_RESEARCH_ -``` - -For instance if the app needed to access `ENV["RAILS_ENV"]` in production we would add this as the key: - -``` -APP_ENV_PROD_RAILS_ENV -``` - -**Important!** -To propagate these new environment variables to the live environments we must finally deploy them manually. [Find the most recent passing deploy](https://github.com/DFE-Digital/buy-for-your-school/actions/workflows/deploy.yml) for the environment you want to provision the new variable on, view it and select "re-run jobs". - -### Infrastructure - -We can also make variables available for Terraform so that it can be configured. - -Here's an example of how `_TF_VAR` is used together with environment prefixes to pass `SYSLOG_DRAIN_URL` information. In this case it configures where logs should be forwarded to at the infrastructure level: - -``` -PREVIEW_TF_VAR_SYSLOG_DRAIN_URL -``` - -## Viewing a variable - -There is no way to view variables through GitHub Secrets once set. - -Though the GitHub secret API could be used, the easiest way to get application variables is to [access the console for the environment you want](console-access.md) and ask Rails for it over the console. - -``` -$irb> ENV["RAILS_ENV"] -``` - -## Checking parity with `bin/climate` - -Given there are multiple remote enviroments (staging, preview, research and production at time of writing), it's possible for some variables to be missing in some of them. - -`bin/climate` provides a few helpful methods for checking things like this: - -### `bin/climate` output - -```console -$ bin/climate - -Commands: - climate envs # Lists environments with a link to edit - climate exists # Given a ENV key, will check which environments have it defined - climate help [COMMAND] # Describe available commands or one specific command - climate parity # Lists Keys that are missing for each environment -``` - -### `bin/climate envs` output - -```console -$ bin/climate envs - -staging: https://github.com/DFE-Digital/buy-for-your-school/settings/environments/241534224/edit -preview: https://github.com/DFE-Digital/buy-for-your-school/settings/environments/241534180/edit -research: https://github.com/DFE-Digital/buy-for-your-school/settings/environments/241530720/edit -production: https://github.com/DFE-Digital/buy-for-your-school/settings/environments/241530368/edit -``` - -### `bin/climate parity` output - -```console -$ bin/climate parity - -## Keys missing in STAGING (https://github.com/DFE-Digital/buy-for-your-school/settings/environments/241534224/edit) -APP_ENV_STAGING_DFE_SIGN_IN_API_ENDPOINT -APP_ENV_STAGING_DFE_SIGN_IN_API_SECRET -APP_ENV_STAGING_GOOGLE_ANALYTICS - -## Keys missing in PREVIEW (https://github.com/DFE-Digital/buy-for-your-school/settings/environments/241534180/edit) -APP_ENV_PREVIEW_CONTENTFUL_ENTRY_CACHING_TTL -APP_ENV_PREVIEW_CONTENTFUL_URL -APP_ENV_PREVIEW_DAYS_A_JOURNEY_CAN_BE_INACTIVE_FOR -APP_ENV_PREVIEW_GOOGLE_ANALYTICS - -## Keys missing in RESEARCH (https://github.com/DFE-Digital/buy-for-your-school/settings/environments/241530720/edit) -APP_ENV_RESEARCH_DAYS_A_JOURNEY_CAN_BE_INACTIVE_FOR -APP_ENV_RESEARCH_DOMAIN -APP_ENV_RESEARCH_RACK_ENV -APP_ENV_RESEARCH_GOOGLE_ANALYTICS - -## Keys missing in PRODUCTION (https://github.com/DFE-Digital/buy-for-your-school/settings/environments/241530368/edit) -APP_ENV_PROD_DAYS_A_JOURNEY_CAN_BE_INACTIVE_FOR - -``` - -### `bin/climate exists` output - -```console -$ bin/climate exists CONTENTFUL_SPACE - -Staging: not present -Preview: not present -Research: not present -Production has CONTENTFUL_SPACE -``` +Add the new environment variables to the variable `application_env` and then deploy the environment. diff --git a/doc/release-process.md b/doc/release-process.md index adae30ae1..9bfa5de32 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -1,55 +1,56 @@ # Release process -To create a new release and deploy it to production, follow this process: +- [Production](#production) +- [Staging](#staging) +- [Development](#development) -## 1. Create the release pull request +## Production -1. Make a branch from `develop` called `release-xxx` (where `xxx` is the sequential release number) -1. Move all features from the `Unreleased` section of [`CHANGELOG.md`](../CHANGELOG.md) to a new heading with the release number linked to a diff of the two latest versions, together with the date in the following format: +### 1. Check what will be released - ```markdown - ## [Release XXX] - 2020-10-27 +To do this, get the diff between `main` and `production` branches. [Comapare here](https://github.com/DFE-Digital/buy-for-your-school/compare/production...main). - ... +### 2. Start the deployment process - [release xxx]: - https://github.com/DFE-Digital/buy-for-your-school/compare/previous-release...release-xxx - ``` +Production deployment is automated with github actions, but you will need to update the `production` branch for it to deploy changes. -1. Create a commit for the release, including the changes for the release in the commit message -1. Push the branch -1. Open a pull request to merge the new branch to `develop` and get it reviewed +Switch to the `production` branch, ensure it is updated from origin, then rebase it with `main`. -## 2. Confirm the release and review the pull request +``` +$ git checkout main +$ git reset --hard origin/main +$ git checkout production +$ git reset --hard origin/production +$ git rebase main +$ git push --force-with-lease +``` -The pull request should be reviewed to confirm that the changes currently are safe to ship and that [`CHANGELOG.md`](../CHANGELOG.md) accurately reflects the changes included in the release: +NOTE: `git push --force-with-lease` is important, the force push will be rejected if changes have been made in orgin that differ from your local branch. -1. Confirm the release with any relevant people (for example the product owner) -1. Think about any dependencies that also need considering: dependent parts of the service that also need updating; environment variables that need changing/adding; third-party services that need to be set up/updated -1. Merge the pull request with `develop` -1. Merge the resultant merge commit into `main` +Monitor the [github actions page](https://github.com/DFE-Digital/buy-for-your-school/actions/workflows/ci-full-pipeline.yml) to check status of the automated deployment. -## 3. Push the tag -Once the pull request has been merged, create a tag against the `main` branch commit in the format `release-xxx` (zero-padded again) and push it to GitHub: +## Staging -```sh -git tag release-xxx merge-commit-for-release -git push origin refs/tags/release-xxx -``` +Staging is automatically deployed with changes appear in `main`. -## 4. Validate the deployment +Monitor the [github actions page](https://github.com/DFE-Digital/buy-for-your-school/actions/workflows/ci-full-pipeline.yml) to check status of the automated deployment. -Check in the production environment to make sure the deployment has gone as expected, and that no problems have arisen during build or release stages. +## Development -## 5. Update Trello +Development deployment is automated with github actions, but you will need to update the `development` branch for it to deploy changes. -Features in the "Approved" column should be moved to "Done". +Switch to the `development` branch, ensure it is updated from origin, then rebase it with `main`. -## 6. Announce the release in Slack +``` +$ git checkout development +$ git reset --hard origin/development +$ git rebase your-feature-branch +$ git push --force-with-lease +``` -Let everybody know that a new release has been shipped. +NOTE: `git push --force-with-lease` is important, the force push will be rejected if changes have been made in orgin that differ from your local branch. This ensures you don't overwrite other developer's changes on development. -The usual place to do this is #sct-buy-for-your-school, with a message like: +If you need to make repeated changes, always reset `development` and then rebase your feature branch again. -> 🚢 Release ### for Buy for your school is now live. (Changes in this release)[https://github.com/DFE-Digital/buy-for-your-school/blob/develop/CHANGELOG.md#release-###---YYYY-MM-DD] - (Production environment)[https://dfe-buy-for-your-school-beta.herokuapp.com/] 🚀 +Monitor the [github actions page](https://github.com/DFE-Digital/buy-for-your-school/actions/workflows/ci-full-pipeline.yml) to check status of the automated deployment. diff --git a/doc/roles-and-portals.md b/doc/roles-and-portals.md new file mode 100644 index 000000000..89543ab73 --- /dev/null +++ b/doc/roles-and-portals.md @@ -0,0 +1,44 @@ +# Roles and Portals + +Access to portals is dictated by the agents roles. The roles are checked on page load and the agent will be redirected if they do not have sufficient roles required to access (as definied in the policy). + +|Portal|Endpoint|Roles required (either)| +|-|-|-| +|Support / ProcOps CMS|`/support`|`internal procops_admin procops`| +|Frameworks Evalutaion|`/frameworks`|`framework_evaluator_admin framework_evaluator`| +|Engagement & Outreach|`/engagement`|`internal e_and_o_admin e_and_o`| + +- See `app/models/support/agent.rb` for full list of roles. +- See `app/policies/cms_portal_policies.rb` for up to date access policy information. +- Agents with role `global_admin` can access any portal. + +## Assigning Roles + +Roles can be changed by an agent with sufficiently high enough roles [see permissions](#role-assigning-permissions). + +To assign roles, visit any of the following management pages: + +|Portal|Role Management URL| +|-|-| +|Support|`/support/management/agents`| +|Engagement|`/engagement/management/agents`| + +Developers can modify the roles array on an agent directly using the rails console or database access. + +``` +$ bundle exec rails c +Running via Spring preloader in process 93412 +Loading development environment (Rails 7.0.8) +[1] pry(main)> Support::Agent.find("xxxx").update(roles: ["new", "roles", "here"]) +``` + +### Role assigning permissions + +|Role|Can assign...| +|-|-| +|Global Admin|Any role| +|Proc-Ops Admin|Proc-Ops staff| +|E & O Admin|E & O staff| +|Framework Evaluator Admin|Framework Evaluator staff| + +Any roles not listed cannot grant roles at all.