Skip to content

Commit

Permalink
merge main
Browse files Browse the repository at this point in the history
  • Loading branch information
Riceyo committed Aug 2, 2024
2 parents 8b93859 + 41db81f commit 61681a2
Show file tree
Hide file tree
Showing 116 changed files with 1,837 additions and 1,705 deletions.
9 changes: 5 additions & 4 deletions .github/workflows/psscriptanalyzer.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
name: Run code checker
on:
pull_request:
branches: [ main ]
branches: [main]
push:
branches: [ main ]
branches: [main]
jobs:
psscriptanalyzer:
name: Run PSScriptAnalyzer
Expand All @@ -13,5 +14,5 @@ jobs:
uses: actions/checkout@v3
- name: Run PSScriptAnalyzer command
shell: pwsh
run: |
Invoke-ScriptAnalyzer -Path .\scripts\ -ExcludeRule PSAvoidUsingPositionalParameters, PSAvoidUsingWriteHost -Recurse
run: |-
Invoke-ScriptAnalyzer -Path .\scripts\ -ExcludeRule PSAvoidUsingPositionalParameters, PSAvoidUsingWriteHost -Recurse
7 changes: 4 additions & 3 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
name: Run unit tests
on:
pull_request:
branches: [ main ]
branches: [main]
push:
branches: [ main ]
branches: [main]
jobs:
pester:
name: Run Pester
Expand All @@ -13,5 +14,5 @@ jobs:
uses: actions/checkout@v3
- name: Run Pester command
shell: pwsh
run: |
run: |-
Invoke-Pester .\scripts\* -Passthru
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.
36 changes: 10 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@

We use Azure DevOps pipelines to build and deploy C# applications, and execute scripts on Windows VMs. Blaise is Windows based and provides a .NET Framework API. We provision VMs via GCP and install an Azure DevOps agent on them. This allows us to deploy our applications onto the VMs and execute any necessary scripts. We call the Azure DevOps pipelines from Concourse via an authenicated HTTPS request.

Azure DevOps is integrated with our GitHub repositories. Changes to the pipeline yaml configuration files in the repositories, will be reflected within Azure DevOps.
Azure DevOps is integrated with our GitHub repositories. Changes to the pipeline YAML configuration files in the repositories, will be reflected within Azure DevOps.

Some of the Azure DevOps pipeline yaml configuration files are stored in the repositories they relate to, such as the [rest-api]() and [nuget](). This resposotory stores Azure DevOps pipeline yaml configuration files that don't directly relate to a service, such as configuring Blaise and running integration tests.
Some of the Azure DevOps pipeline YAML configuration files are stored in the repositories they relate to, such as the [rest-api](https://github.com/ONSdigital/blaise-api-rest) and [nuget-api](https://github.com/ONSdigital/blaise-nuget-api). This repository, however, stores Azure DevOps pipeline YAML configuration files that don't directly relate to a specific service, such as those for configuring Blaise, running integration tests, and deploying services.

Azure DevOps pipeline require at least the following parameters:

- `VarGroup` - Contains various environment variables and is created by [Terraform](). It's usually the GCP project name.
- `VarGroup` - Contains various environment variables and is created by [Terraform](https://github.com/ONSdigital/blaise-terraform). It's usually the GCP project name.
- `Environment` - Informs Azure DevOps which VMs to execute the pipeline on. Created manually in the Azure DevOps web UI and is `dev`, `preprod`, and `prod` for the formal environments. Sandboxes is usually the developers first name.

## Environment deployment tags

VMs (Virual Machines) are labelled with tags via a startup script when the Azure DevOps agent is registered with the VM. The VM startup scripts are stored in the [Terraform]() respository. Tags are used to target specific VMs via the yaml, the following snippet shows how to deploy to all VMs with the tag `data-entry`.
VMs (Virual Machines) are labelled with tags via a startup script when the Azure DevOps agent is registered with the VM. The VM startup scripts are stored in the [Terraform](https://github.com/ONSdigital/blaise-terraform) respository. Tags are used to target specific VMs via the YAML, the following snippet shows how to deploy to all VMs with the tag `data-entry`.

```
environment:
Expand All @@ -24,7 +24,7 @@ environment:

## Hosted environemnt deployment

Not all deployments are targetted at our VMs in GCP. Integration tests for example are run from VMs hosted by Azure DevOps. Example yaml snippet:
Not all deployments are targetted at our VMs in GCP. Somne of the integration tests for example are run from VMs hosted by Azure DevOps. Example YAML snippet:

```
pool:
Expand All @@ -33,7 +33,7 @@ pool:

## Templates

Reusable yaml steps are created within the templates folder, task step format as follows:
Reusable YAML steps are created within the templates folder, task step format as follows:

```
steps:
Expand All @@ -43,7 +43,7 @@ steps:
implementation
```

To use a template within a yaml file:
To use a template within a YAML file:

```
- template: /templates/my_template.yml
Expand All @@ -67,24 +67,8 @@ az pipelines create --name "A name for your pipeline" --yml-path pipelines/pipel
1. Navigate to https://dev.azure.com and login with your ONS email
1. Go to *pipelines*
1. Click *New pipeline*
1. Select *Github (YAML)*
1. Select *ONSDigital/Blaise-Azure-Pipelines* repo
1. Select *GitHub (YAML)*
1. Select *ONSDigital/blaise-azure-pipelines* repo
1. Select *Existing Azure Pipelines YAML File*
1. Select your yaml file in *Path* - If you are working from a branch change the Branch to point at that (by default Azure will always look at main, so you will not need to redo this when you merge)
1. Select your YAML file in *Path* - If you are working from a branch, update the branch reference accordingly. By default, Azure will always point to the main branch, so this change will not need to be redone after merging.
1. Save the pipeline

### Create a new user role
1. Add the permissions to the `UserRoles.json` file in the `UserRoles` folder
2. Ensure that the new role does not contain `root` (see example below) as this will prevent it from propagating to CATI
```angular2html
{
"name":"TO Appointments",
"description":"Role for TO Appointments",
"permissions":[
"root",
"CATI",
"CATI.viewappointments",
"CATI.selectfromcaseinfo"
]
},
```
6 changes: 6 additions & 0 deletions pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@
Installs Blaise, installs database connector, opens ports on the VMs, sets up Blaise, and sets up the databases Blaise will use.

Note that the databases won't be setup again if their Blaise data interface file already exist on the management VM. This is because running the setup again will wipe any data on the database!

If the installed Blaise version differs from the version specified in the ENV_BLAISE_CURRENT_VERSION environment variable, the current Blaise version will be uninstalled, and the required version will be installed. The scripts will attempt to delete the dashboard IIS folders, as these can cause issues when changing versions. However, this process isn't always successful and may need to be done manually. Changing the Blaise version can sometimes result in the deletion of essential questionnaire files, such as BMIXs, which may need to be restored. Additionally, you will likely need to access Blaise Server Manager in admin mode to verify that the server roles are set correctly and re-commit them if necessary.

# Configure User Roles

When creating a new user role, ensure that the `root` permission is *NOT* given. This prevents the other permissions propagating.
72 changes: 33 additions & 39 deletions pipelines/blaise_smoke_tests.yml
Original file line number Diff line number Diff line change
@@ -1,48 +1,42 @@
---
parameters:
- name: VarGroup
displayName: Variable Group
displayName: Variable group
- name: Environment
displayName: Environment to use
- name: AutomatedTestsBranch
displayName: Automated tests git branch
default: master

- name: IntegrationTestsBranch
displayName: Branch to use
default: main
trigger: none

pr: none

variables:
- group: ${{ parameters.VarGroup }}
- group: ${{parameters.VarGroup}}
- template: /templates/variables.yml

stages:
- stage: BlaiseInstall_${{parameters.Environment }}
displayName: ${{parameters.Environment }} Installation Of Blaise
jobs:
- job: "SmokeTests"
variables:
ENV_BLAISE_SERVER_BINDING : $(ENV_BLAISE_EXTERNAL_SERVER_BINDING)
ENV_BLAISE_SERVER_HOST_NAME : $(ENV_BLAISE_EXTERNAL_SERVER_HOST_NAME)
pool:
vmImage: 'windows-2019'
steps:
- template: /templates/blaise_reg_settings.yml
parameters:
BlaiseLicenseKey: $(ENV_BLAISE_LICENSE_KEY)
BlaiseActivationKey: $(ENV_BLAISE_ACTIVATION_CODE)

- template: /templates/download_test_instrument.yml

- template: /templates/download_artifact_steps.yml
parameters:
GitBranch: ${{parameters.AutomatedTestsBranch}}
ArtifactName: _AutomatedTests
DownloadPath: $(System.ArtifactsDirectory)
TargetFiles: '**/*Behaviour*.config'
PipelineNumber: 43

- template: /templates/automated_test_with_category_steps.yml
parameters:
BehaviourSolutionName: Blaise.Tests.Behaviour
TestName: Blaise Smoke Tests
TestCategory: Smoke
- stage: BlaiseTests_${{parameters.Environment}}
displayName: Blaise tests (${{parameters.Environment}})
jobs:
- job: BlaiseTests
variables:
ENV_BLAISE_SERVER_BINDING: $(ENV_BLAISE_EXTERNAL_SERVER_BINDING)
ENV_BLAISE_SERVER_HOST_NAME: $(ENV_BLAISE_EXTERNAL_SERVER_HOST_NAME)
pool:
vmImage: windows-2019
steps:
- template: /templates/set_license_key.yml
parameters:
BlaiseLicenseKey: $(ENV_BLAISE_LICENSE_KEY)
BlaiseActivationKey: $(ENV_BLAISE_ACTIVATION_CODE)
- template: /templates/download_test_questionnaire.yml
- template: /templates/download_build_artifact.yml
parameters:
Branch: ${{parameters.IntegrationTestsBranch}}
ArtifactName: _AutomatedTests
DownloadPath: $(System.ArtifactsDirectory)
TargetFiles: '**/*Behaviour*.config'
PipelineNumber: 43
- template: /templates/run_tests_with_category.yml
parameters:
TestSolutionName: Blaise.Tests.Behaviour
TestName: Blaise smoke tests
TestCategory: smoke
87 changes: 39 additions & 48 deletions pipelines/cati_automated_tests.yml
Original file line number Diff line number Diff line change
@@ -1,57 +1,48 @@
---
parameters:
- name: VarGroup
displayName: Variable Group
displayName: Variable group
- name: Environment
displayName: Environment to use
- name: AutomatedTestsBranch
displayName: Automated tests git branch
default: master

- name: IntegrationTestsBranch
displayName: Branch to use
default: main
trigger: none

pr: none

variables:
- group: ${{ parameters.VarGroup }}
- group: ${{parameters.VarGroup}}
- template: /templates/variables.yml

stages:
- stage: CATITests_${{parameters.Environment }}
displayName: ${{parameters.Environment }} CATI Automated Tests
jobs:
- job: "AutomatedTests"
timeoutInMinutes: 0 # how long to run the job before automatically cancelling
cancelTimeoutInMinutes: 2 # how much time to give 'run always even if cancelled tasks' before stopping them
variables:
ENV_BLAISE_SERVER_BINDING : $(ENV_BLAISE_EXTERNAL_SERVER_BINDING)
ENV_BLAISE_SERVER_HOST_NAME : $(ENV_BLAISE_EXTERNAL_SERVER_HOST_NAME)
pool:
vmImage: 'windows-2019'
steps:
- template: /templates/blaise_reg_settings.yml
parameters:
BlaiseLicenseKey: $(ENV_BLAISE_LICENSE_KEY)
BlaiseActivationKey: $(ENV_BLAISE_ACTIVATION_CODE)

- template: /templates/download_test_instrument.yml

- template: /templates/download_artifact_steps.yml
parameters:
GitBranch: ${{parameters.AutomatedTestsBranch}}
ArtifactName: _AutomatedTests
DownloadPath: $(System.ArtifactsDirectory)
TargetFiles: '**/*Behaviour*.config'
PipelineNumber: 43

- ${{ if notin(parameters.Environment, 'dev', 'preprod') }}:
- template: /templates/automated_test_with_category_steps.yml
parameters:
BehaviourSolutionName: Blaise.CATI.Tests.Behaviour
TestName: Cati Smoke Tests
TestCategory: Smoke

- ${{ if in(parameters.Environment, 'dev', 'preprod') }}:
- template: /templates/automated_test_steps.yml
parameters:
BehaviourSolutionName: Blaise.CATI.tests.Behaviour
TestName: Full Cati Regression Test
- stage: CatiTests_${{parameters.Environment}}
displayName: CATI tests (${{parameters.Environment}})
jobs:
- job: CatiTests
variables:
ENV_BLAISE_SERVER_BINDING: $(ENV_BLAISE_EXTERNAL_SERVER_BINDING)
ENV_BLAISE_SERVER_HOST_NAME: $(ENV_BLAISE_EXTERNAL_SERVER_HOST_NAME)
pool:
vmImage: windows-2019
steps:
- template: /templates/set_license_key.yml
parameters:
BlaiseLicenseKey: $(ENV_BLAISE_LICENSE_KEY)
BlaiseActivationKey: $(ENV_BLAISE_ACTIVATION_CODE)
- template: /templates/download_test_questionnaire.yml
- template: /templates/download_build_artifact.yml
parameters:
Branch: ${{parameters.IntegrationTestsBranch}}
ArtifactName: _AutomatedTests
DownloadPath: $(System.ArtifactsDirectory)
TargetFiles: '**/*Behaviour*.config'
PipelineNumber: 43
- ${{ if eq(parameters.Environment, 'prod') }}:
- template: /templates/run_tests_with_category.yml
parameters:
TestSolutionName: Blaise.CATI.Tests.Behaviour
TestName: CATI smoke tests
TestCategory: smoke
- ${{ if ne(parameters.Environment, 'prod') }}:
- template: /templates/run_tests.yml
parameters:
TestSolutionName: Blaise.CATI.Tests.Behaviour
TestName: CATI regression tests
Loading

0 comments on commit 61681a2

Please sign in to comment.