Skip to content

Commit

Permalink
Merge pull request #157 from ONSdigital/BLAIS5-4295-refactor
Browse files Browse the repository at this point in the history
Promote from BLAIS5-4295-refactor to main
  • Loading branch information
Riceyo authored Jul 30, 2024
2 parents 87d2dc4 + 736d4f1 commit 0781c20
Show file tree
Hide file tree
Showing 112 changed files with 1,566 additions and 1,564 deletions.
9 changes: 5 additions & 4 deletions .github/workflows/psscriptanalyzer.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
name: Run code checker
on:
pull_request:
branches: [ main ]
branches: [main]
push:
branches: [ main ]
branches: [main]
jobs:
psscriptanalyzer:
name: Run PSScriptAnalyzer
Expand All @@ -13,5 +14,5 @@ jobs:
uses: actions/checkout@v3
- name: Run PSScriptAnalyzer command
shell: pwsh
run: |
Invoke-ScriptAnalyzer -Path .\scripts\ -ExcludeRule PSAvoidUsingPositionalParameters, PSAvoidUsingWriteHost -Recurse
run: |-
Invoke-ScriptAnalyzer -Path .\scripts\ -ExcludeRule PSAvoidUsingPositionalParameters, PSAvoidUsingWriteHost -Recurse
7 changes: 4 additions & 3 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
name: Run unit tests
on:
pull_request:
branches: [ main ]
branches: [main]
push:
branches: [ main ]
branches: [main]
jobs:
pester:
name: Run Pester
Expand All @@ -13,5 +14,5 @@ jobs:
uses: actions/checkout@v3
- name: Run Pester command
shell: pwsh
run: |
run: |-
Invoke-Pester .\scripts\* -Passthru
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@

We use Azure DevOps pipelines to build and deploy C# applications, and execute scripts on Windows VMs. Blaise is Windows based and provides a .NET Framework API. We provision VMs via GCP and install an Azure DevOps agent on them. This allows us to deploy our applications onto the VMs and execute any necessary scripts. We call the Azure DevOps pipelines from Concourse via an authenicated HTTPS request.

Azure DevOps is integrated with our GitHub repositories. Changes to the pipeline yaml configuration files in the repositories, will be reflected within Azure DevOps.
Azure DevOps is integrated with our GitHub repositories. Changes to the pipeline YAML configuration files in the repositories, will be reflected within Azure DevOps.

Some of the Azure DevOps pipeline yaml configuration files are stored in the repositories they relate to, such as the [rest-api]() and [nuget](). This resposotory stores Azure DevOps pipeline yaml configuration files that don't directly relate to a service, such as configuring Blaise and running integration tests.
Some of the Azure DevOps pipeline YAML configuration files are stored in the repositories they relate to, such as the [rest-api](https://github.com/ONSdigital/blaise-api-rest) and [nuget-api](https://github.com/ONSdigital/blaise-nuget-api). This repository, however, stores Azure DevOps pipeline YAML configuration files that don't directly relate to a specific service, such as those for configuring Blaise, running integration tests, and deploying services.

Azure DevOps pipeline require at least the following parameters:

- `VarGroup` - Contains various environment variables and is created by [Terraform](). It's usually the GCP project name.
- `VarGroup` - Contains various environment variables and is created by [Terraform](https://github.com/ONSdigital/blaise-terraform). It's usually the GCP project name.
- `Environment` - Informs Azure DevOps which VMs to execute the pipeline on. Created manually in the Azure DevOps web UI and is `dev`, `preprod`, and `prod` for the formal environments. Sandboxes is usually the developers first name.

## Environment deployment tags

VMs (Virual Machines) are labelled with tags via a startup script when the Azure DevOps agent is registered with the VM. The VM startup scripts are stored in the [Terraform]() respository. Tags are used to target specific VMs via the yaml, the following snippet shows how to deploy to all VMs with the tag `data-entry`.
VMs (Virual Machines) are labelled with tags via a startup script when the Azure DevOps agent is registered with the VM. The VM startup scripts are stored in the [Terraform](https://github.com/ONSdigital/blaise-terraform) respository. Tags are used to target specific VMs via the YAML, the following snippet shows how to deploy to all VMs with the tag `data-entry`.

```
environment:
Expand All @@ -24,7 +24,7 @@ environment:

## Hosted environemnt deployment

Not all deployments are targetted at our VMs in GCP. Integration tests for example are run from VMs hosted by Azure DevOps. Example yaml snippet:
Not all deployments are targetted at our VMs in GCP. Somne of the integration tests for example are run from VMs hosted by Azure DevOps. Example YAML snippet:

```
pool:
Expand All @@ -33,7 +33,7 @@ pool:

## Templates

Reusable yaml steps are created within the templates folder, task step format as follows:
Reusable YAML steps are created within the templates folder, task step format as follows:

```
steps:
Expand All @@ -43,7 +43,7 @@ steps:
implementation
```

To use a template within a yaml file:
To use a template within a YAML file:

```
- template: /templates/my_template.yml
Expand All @@ -70,5 +70,5 @@ az pipelines create --name "A name for your pipeline" --yml-path pipelines/pipel
1. Select *GitHub (YAML)*
1. Select *ONSDigital/blaise-azure-pipelines* repo
1. Select *Existing Azure Pipelines YAML File*
1. Select your yaml file in *Path* - If you are working from a branch, update the branch reference accordingly. By default, Azure will always point to the main branch, so this change will not need to be redone after merging.
1. Select your YAML file in *Path* - If you are working from a branch, update the branch reference accordingly. By default, Azure will always point to the main branch, so this change will not need to be redone after merging.
1. Save the pipeline
72 changes: 33 additions & 39 deletions pipelines/blaise_smoke_tests.yml
Original file line number Diff line number Diff line change
@@ -1,48 +1,42 @@
---
parameters:
- name: VarGroup
displayName: Variable Group
displayName: Variable group
- name: Environment
displayName: Environment to use
- name: AutomatedTestsBranch
displayName: Automated tests git branch
default: master

- name: IntegrationTestsBranch
displayName: Branch to use
default: main
trigger: none

pr: none

variables:
- group: ${{ parameters.VarGroup }}
- group: ${{parameters.VarGroup}}
- template: /templates/variables.yml

stages:
- stage: BlaiseInstall_${{parameters.Environment }}
displayName: ${{parameters.Environment }} Installation Of Blaise
jobs:
- job: "SmokeTests"
variables:
ENV_BLAISE_SERVER_BINDING : $(ENV_BLAISE_EXTERNAL_SERVER_BINDING)
ENV_BLAISE_SERVER_HOST_NAME : $(ENV_BLAISE_EXTERNAL_SERVER_HOST_NAME)
pool:
vmImage: 'windows-2019'
steps:
- template: /templates/blaise_reg_settings.yml
parameters:
BlaiseLicenseKey: $(ENV_BLAISE_LICENSE_KEY)
BlaiseActivationKey: $(ENV_BLAISE_ACTIVATION_CODE)

- template: /templates/download_test_instrument.yml

- template: /templates/download_artifact_steps.yml
parameters:
GitBranch: ${{parameters.AutomatedTestsBranch}}
ArtifactName: _AutomatedTests
DownloadPath: $(System.ArtifactsDirectory)
TargetFiles: '**/*Behaviour*.config'
PipelineNumber: 43

- template: /templates/automated_test_with_category_steps.yml
parameters:
BehaviourSolutionName: Blaise.Tests.Behaviour
TestName: Blaise Smoke Tests
TestCategory: Smoke
- stage: BlaiseTests_${{parameters.Environment}}
displayName: Blaise tests (${{parameters.Environment}})
jobs:
- job: BlaiseTests
variables:
ENV_BLAISE_SERVER_BINDING: $(ENV_BLAISE_EXTERNAL_SERVER_BINDING)
ENV_BLAISE_SERVER_HOST_NAME: $(ENV_BLAISE_EXTERNAL_SERVER_HOST_NAME)
pool:
vmImage: windows-2019
steps:
- template: /templates/set_license_key.yml
parameters:
BlaiseLicenseKey: $(ENV_BLAISE_LICENSE_KEY)
BlaiseActivationKey: $(ENV_BLAISE_ACTIVATION_CODE)
- template: /templates/download_test_questionnaire.yml
- template: /templates/download_build_artifact.yml
parameters:
Branch: ${{parameters.IntegrationTestsBranch}}
ArtifactName: _AutomatedTests
DownloadPath: $(System.ArtifactsDirectory)
TargetFiles: '**/*Behaviour*.config'
PipelineNumber: 43
- template: /templates/run_tests_with_category.yml
parameters:
TestSolutionName: Blaise.Tests.Behaviour
TestName: Blaise smoke tests
TestCategory: smoke
87 changes: 39 additions & 48 deletions pipelines/cati_automated_tests.yml
Original file line number Diff line number Diff line change
@@ -1,57 +1,48 @@
---
parameters:
- name: VarGroup
displayName: Variable Group
displayName: Variable group
- name: Environment
displayName: Environment to use
- name: AutomatedTestsBranch
displayName: Automated tests git branch
default: master

- name: IntegrationTestsBranch
displayName: Branch to use
default: main
trigger: none

pr: none

variables:
- group: ${{ parameters.VarGroup }}
- group: ${{parameters.VarGroup}}
- template: /templates/variables.yml

stages:
- stage: CATITests_${{parameters.Environment }}
displayName: ${{parameters.Environment }} CATI Automated Tests
jobs:
- job: "AutomatedTests"
timeoutInMinutes: 0 # how long to run the job before automatically cancelling
cancelTimeoutInMinutes: 2 # how much time to give 'run always even if cancelled tasks' before stopping them
variables:
ENV_BLAISE_SERVER_BINDING : $(ENV_BLAISE_EXTERNAL_SERVER_BINDING)
ENV_BLAISE_SERVER_HOST_NAME : $(ENV_BLAISE_EXTERNAL_SERVER_HOST_NAME)
pool:
vmImage: 'windows-2019'
steps:
- template: /templates/blaise_reg_settings.yml
parameters:
BlaiseLicenseKey: $(ENV_BLAISE_LICENSE_KEY)
BlaiseActivationKey: $(ENV_BLAISE_ACTIVATION_CODE)

- template: /templates/download_test_instrument.yml

- template: /templates/download_artifact_steps.yml
parameters:
GitBranch: ${{parameters.AutomatedTestsBranch}}
ArtifactName: _AutomatedTests
DownloadPath: $(System.ArtifactsDirectory)
TargetFiles: '**/*Behaviour*.config'
PipelineNumber: 43

- ${{ if notin(parameters.Environment, 'dev', 'preprod') }}:
- template: /templates/automated_test_with_category_steps.yml
parameters:
BehaviourSolutionName: Blaise.CATI.Tests.Behaviour
TestName: Cati Smoke Tests
TestCategory: Smoke

- ${{ if in(parameters.Environment, 'dev', 'preprod') }}:
- template: /templates/automated_test_steps.yml
parameters:
BehaviourSolutionName: Blaise.CATI.tests.Behaviour
TestName: Full Cati Regression Test
- stage: CatiTests_${{parameters.Environment}}
displayName: CATI tests (${{parameters.Environment}})
jobs:
- job: CatiTests
variables:
ENV_BLAISE_SERVER_BINDING: $(ENV_BLAISE_EXTERNAL_SERVER_BINDING)
ENV_BLAISE_SERVER_HOST_NAME: $(ENV_BLAISE_EXTERNAL_SERVER_HOST_NAME)
pool:
vmImage: windows-2019
steps:
- template: /templates/set_license_key.yml
parameters:
BlaiseLicenseKey: $(ENV_BLAISE_LICENSE_KEY)
BlaiseActivationKey: $(ENV_BLAISE_ACTIVATION_CODE)
- template: /templates/download_test_questionnaire.yml
- template: /templates/download_build_artifact.yml
parameters:
Branch: ${{parameters.IntegrationTestsBranch}}
ArtifactName: _AutomatedTests
DownloadPath: $(System.ArtifactsDirectory)
TargetFiles: '**/*Behaviour*.config'
PipelineNumber: 43
- ${{ if eq(parameters.Environment, 'prod') }}:
- template: /templates/run_tests_with_category.yml
parameters:
TestSolutionName: Blaise.CATI.Tests.Behaviour
TestName: CATI smoke tests
TestCategory: smoke
- ${{ if ne(parameters.Environment, 'prod') }}:
- template: /templates/run_tests.yml
parameters:
TestSolutionName: Blaise.CATI.Tests.Behaviour
TestName: CATI regression tests
Loading

0 comments on commit 0781c20

Please sign in to comment.