Skip to content

Commit

Permalink
🚀 Dev/Test deployment changes (#87)
Browse files Browse the repository at this point in the history
* NIT-880 deploy only transform and needed dependencies

* NIT-934 deploying content services in cloud platform

* NIT-934 deploying content services in cloud platform

* NIT-934 deploying content services in cloud platform

* NIT-934 deploying content services in cloud platform

* NIT-934 deploying content services in cloud platform

* NIT-935 deploying share service in cloud platform

* NIT-935 deploying share service in cloud platform

* NIT-953 using external hostname for the share component

* NIT-948 base64 encoded secret instead of plain text for solr/repo communication

* NIT-938 deleted charts subdirectories that were not needed

* NIT-938 deleted charts subdirectories that were not needed

* NIT-938 deleted charts subdirectories that were not needed

* NIT-938 Instuctions added on how to modify and publish a chart to github pages

* NIT-948 Updagetd readme for shared secret extraction

* Update README.md

* NIT-938 Updagetd readme for shared secret extraction

* NIT-938 update the readme for starting services

* NIT-952 initial workflow version

* NIT-952 point at share dockerfile

* NIT-952 replace share image repository

* NIT-979 Config updated to ensure redeployed services do restart

* NIT-969 Ensuring s3 directory structure is correct

* NIT-990 Using updated action variables and secrets

* NIT-990 Using a new container tag that resulted from updating action variables and secrets

* NIT-990 Using pipeline generated image

* NIT-990 Updated number of replicas for each service as advised by ALF dev team

* NIT-989 use alf license provided through a secret

* NIT-966 use namespace name to build service url/hostname

* NIT-994 pipeline for installing helm charts (#20)

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 pipeline for installing helm charts

* NIT-994 Adding matrix strategy

* NIT-994 Adding matrix strategy

* NIT-994 Adding matrix strategy

* NIT-994 Adding matrix strategy

* NIT-994 Adding matrix strategy

* NIT-994 Adding matrix strategy

* NIT-994 Adding matrix strategy

* Nit 1015 alfresco move existing share gh workflow to dev repo (#22)

* NIT-1015 Share GH workflow to move to hmpps-alfresco repo

* NIT-1015 Share GH workflow to move to hmpps-alfresco repo

* NIT-1017 disable activiti engine as its not in use

* NIT-1016 Update share image based on the custom code (#23)

* NIT-1016 Update share image based on the custom code

* NIT-1046 allow the pipeline to modify the dev namespace

* NIT-1097 limit access to alfresco envs

* NIT-1151 test env added in pipeline matrix deployments

* NIT-1151 updating whitelist for test env

* NIT-1151 allow internet traffic from cp

* NIT-1098 update names for consistency

* NIT-1098 add refresh jobs and workflow

* NIT-1098 remove forgotten debug options

* NIT-1178 increase size of ephemeral volume in test

* NIT-1178 update defaults to prevent uncontrolled restarts

* NIT-1178 align resource limit closer to alfresco recommendations

* NIT-1170 increase number of repository pods in poc and dev

* NIT-1170 increase number of repository pods in test

* NIT-1170 temporarily disable poc deployments

* NIT-1098 dump and restore db in parallel

* Increase repository and tika resource limits in test

* NIT-1181 adding security context to pods for fixing warnings

* NIT-1181 remove comment

* NIT-1193 enable prop db table cleaner v2

* NIT-1169 data refresh process improvements

* NIT-1169 temporary data migration helpers

* NIT-1169 correct descriptions

* trigger release only on relevant code changes

* NIT-1120 k8s job to restore docs worker

* NIT-1120 updating workflow

* NIT-1120 updating workflow

* NIT-1120 updating workflow

* NIT-1120 updating workflow

* NIT-1120 updating workflow

* NIT-1120 updating workflow

* NIT-1120 updating workflow

* NIT-1120 updating workflow

* NIT-1231 initial config for live-like environments

* NIT-1120 using simple exec to run scripts instead of creating a job

* NIT-1120 using simple exec to run scripts instead of creating a job

* NIT-1120 using simple exec to run scripts instead of creating a job

* NIT-1120 using simple exec to run scripts instead of creating a job

* NIT-1120 using simple exec to run scripts instead of creating a job

* NIT-1120 extract versionId from delete markers after enabling versioning

* new alf release on poc

* optimise test requests and limits

* add workflow

* untar chart

* NIT-1255 use Xms/Xmx for JVM memory allocations

* NIT-1255 set JVM memory values for stage

* Update helm-release.yml

* helm-dependencies

* unarchive

* Update helm-release.yml

* Update helm-release.yml

* 🎉 Initial workflow for snapshot creation

* Delete create-pv-restore-point.yml

* compress files to a tar.gz and copy to dedicated bucket

* correct s3 bucket secret key

* remove yaml loop

* 🔥 Remove solr backup job

* NIT-1253 add workflow to make Alf env read-only

* NIT-1229 workflows to scale up and down lower envs

* NIT-1229 workflows to scale up and down lower envs

* NIT-1253 change to kubectl annotate method instead of Helm upgrade

* typo

* Workflow name typo

* Update create-rds-snapshot.yml

* Update create-rds-snapshot.yml

* Update create-rds-snapshot.yml

* Update create-rds-snapshot.yml

* Update create-rds-snapshot.yml

* Update create-rds-snapshot.yml

* NIT-1253 final revision

* NIT-1253 don't hardcode namespace

* NIT-1253 implement removal option

* NIT-1253 add validation to ensure new snippet isn't added multipe times

* Give share ingress unique hostname and cert

* NIT-1229 fetching values from helm release

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 change to manual run for testing and using json format

* NIT-1229 changing back to scheduled run

* NIT-1229 workflow to run on weekdays only

* NIT-1256 run long running jobs on runners available in org

* NIT-1279 testing recovery process

* NIT-1279 testing recovery process

* NIT-1279 restore status check and increase in wait time

* NIT-1279 restore status check and increase in wait time

* NIT-1279 passing env variables explicitly

* NIT-1279 check for when deleted marker is not available

* NIT-1279 check for when deleted marker is not available

* NIT-1279 swithing off debug mode and change in exit code

* NIT-1298 adding dev to autoscaling matrix

* NIT-1305 diff scale time for dev

* NIT-1305 dev to scale up at diff times

* NIT-1305 dev to scale up at diff times

* NIT-1305 adding a reusable workflow for scaling up

* NIT-1305 fix error in script

* chore: Add templated directory to .gitignore and update IP whitelist in values files

* Update cloud-platform-deploy-release.yml

* remove share subdomain

* redirect to /share

* NIT-1425 fix typo for workflow

* ✨ Update DB Refresh workflow to include Slack notifications (#76)

* ✨ Update DB Refresh workflow to include Slack notifications

* 🐛 poc now uses opensearch

* Update data-refresh.yaml

* --ignore-not-found for helm uninstall

* Happy Helming

* Update data-refresh.yaml

* 🐛 Correct service account names for refresh jobs

* fix: Add curl package and fix Slack message

* typo

* ✨ create image and update job

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update Dockerfile

Update job.yaml

Update Dockerfile

Update Dockerfile

Update job.yaml

absolute dirs

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update Dockerfile

non root support

Update job.yaml

Update data-refresh.yaml

up

Update build-push-db-utils.yml

rename

Update build-push-db-utils.yml

Update build-push-db-utils.yml

no need to install jq

no password

update jobs for refresh

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* t

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* typo

* Update job.yaml

* Update job.yaml

* Update data-refresh.yaml

---------

Co-authored-by: George Taylor <[email protected]>

* ✨ Update DB Refresh workflow to include Slack notifications

* ✨ Update DB Refresh workflow to include Slack notifications

* 🐛 poc now uses opensearch

* Update data-refresh.yaml

* --ignore-not-found for helm uninstall

* Happy Helming

* Update data-refresh.yaml

* 🐛 Correct service account names for refresh jobs

* fix: Add curl package and fix Slack message

* typo

* ✨ create image and update job

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update Dockerfile

Update job.yaml

Update Dockerfile

Update Dockerfile

Update job.yaml

absolute dirs

Update job.yaml

Update job.yaml

Update job.yaml

Update job.yaml

Update Dockerfile

non root support

Update job.yaml

Update data-refresh.yaml

up

Update build-push-db-utils.yml

rename

Update build-push-db-utils.yml

Update build-push-db-utils.yml

no need to install jq

no password

update jobs for refresh

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* t

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* Update job.yaml

* typo

* Update job.yaml

* Update job.yaml

* Update data-refresh.yaml

---------

Co-Authored-By: George Taylor <[email protected]>

* ✨ Helm chart configuration for Opensearch and enterprise search connector (#70)

* update values for opensearch

* add makefile +chart changes

* Update values.yaml

* Update values.yaml

* update makefile

* Update values.yaml

* Update values.yaml

* Update values.yaml

* vars

* Update values.yaml

* Update values.yaml

* images

* config map

* Revert "config map"

This reverts commit e714ada.

* Reapply "config map"

This reverts commit c47acb8.

* config map

* images

* Squashed commit of the following:

commit 6cda4f5
Merge: a682b79 ca20a8c
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 15:03:01 2024 +0100

    Merge pull request #75 from ministryofjustice/fix-ingress-share-real

    Revert share ingress to dedicated hostname + enable redirect to /share

commit ca20a8c
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 13:15:37 2024 +0100

    redirect to /share

commit a682b79
Merge: 066cecc 1780e23
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 11:23:05 2024 +0100

    Merge pull request #74 from ministryofjustice/remove-share-sub-domain

    Remove share subdomain

commit 1780e23
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 11:21:18 2024 +0100

    remove share subdomain

commit 066cecc
Merge: 4d23e4b b21f656
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:36:49 2024 +0100

    Merge pull request #73 from ministryofjustice/workflow-changes

    Update cloud-platform-deploy-release.yml

commit b21f656
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:35:57 2024 +0100

    Update cloud-platform-deploy-release.yml

commit 4d23e4b
Merge: 6aeb967 ca1cfb2
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:09:57 2024 +0100

    Merge pull request #72 from ministryofjustice/add-airflow-ips

    chore: Add templated directory to .gitignore and update IP whitelist in values files

commit ca1cfb2
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:08:14 2024 +0100

    chore: Add templated directory to .gitignore and update IP whitelist in values files

commit 6aeb967
Merge: f6a7c0e 540fe64
Author: Prem Basumatary <[email protected]>
Date:   Fri Jul 5 09:10:27 2024 +0100

    Merge pull request #71 from ministryofjustice/NIT-1305-custom-scaling-times

    NIT-1305 fix error in script

commit 540fe64
Author: Prem Basumatary <[email protected]>
Date:   Fri Jul 5 09:03:12 2024 +0100

    NIT-1305 fix error in script

* Update ingress-share.yaml

* Use multi live indexing

vals

indexing

Update values.yaml

Update liveindexing-deployment.yaml

Update values.yaml

* ♻️  NIT 1402 alfresco use stock helm chart (#81)

* update values for opensearch

* add makefile +chart changes

* Update values.yaml

* Update values.yaml

* update makefile

* Update values.yaml

* Update values.yaml

* Update values.yaml

* vars

* Update values.yaml

* Update values.yaml

* images

* config map

* Revert "config map"

This reverts commit e714ada.

* Reapply "config map"

This reverts commit c47acb8.

* config map

* images

* Squashed commit of the following:

commit 6cda4f5
Merge: a682b79 ca20a8c
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 15:03:01 2024 +0100

    Merge pull request #75 from ministryofjustice/fix-ingress-share-real

    Revert share ingress to dedicated hostname + enable redirect to /share

commit ca20a8c
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 13:15:37 2024 +0100

    redirect to /share

commit a682b79
Merge: 066cecc 1780e23
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 11:23:05 2024 +0100

    Merge pull request #74 from ministryofjustice/remove-share-sub-domain

    Remove share subdomain

commit 1780e23
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 11:21:18 2024 +0100

    remove share subdomain

commit 066cecc
Merge: 4d23e4b b21f656
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:36:49 2024 +0100

    Merge pull request #73 from ministryofjustice/workflow-changes

    Update cloud-platform-deploy-release.yml

commit b21f656
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:35:57 2024 +0100

    Update cloud-platform-deploy-release.yml

commit 4d23e4b
Merge: 6aeb967 ca1cfb2
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:09:57 2024 +0100

    Merge pull request #72 from ministryofjustice/add-airflow-ips

    chore: Add templated directory to .gitignore and update IP whitelist in values files

commit ca1cfb2
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:08:14 2024 +0100

    chore: Add templated directory to .gitignore and update IP whitelist in values files

commit 6aeb967
Merge: f6a7c0e 540fe64
Author: Prem Basumatary <[email protected]>
Date:   Fri Jul 5 09:10:27 2024 +0100

    Merge pull request #71 from ministryofjustice/NIT-1305-custom-scaling-times

    NIT-1305 fix error in script

commit 540fe64
Author: Prem Basumatary <[email protected]>
Date:   Fri Jul 5 09:03:12 2024 +0100

    NIT-1305 fix error in script

* Update ingress-share.yaml

* vals

* kustomize base

* ingress

* setup makefiel

* ingress poc patches

* correct poc ingress

* Update values.yaml

* update for helm values mergign

* updates

* kustomization base

* rearrange patching for ingress to repeat less

* fix ingress patching with allow list

* :fire: remove alf charts

* rename values files

* :tada: Add kustomize patching for dev environment (#84)

* 🎉 Add kustomize patching for dev environment

* ♻️ refactoring prefixes

* ✨ Uncouple the reindexing job from the main helm charts (#83)

* update values for opensearch

* add makefile +chart changes

* Update values.yaml

* Update values.yaml

* update makefile

* Update values.yaml

* Update values.yaml

* Update values.yaml

* vars

* Update values.yaml

* Update values.yaml

* images

* config map

* Revert "config map"

This reverts commit e714ada.

* Reapply "config map"

This reverts commit c47acb8.

* config map

* images

* Squashed commit of the following:

commit 6cda4f5
Merge: a682b79 ca20a8c
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 15:03:01 2024 +0100

    Merge pull request #75 from ministryofjustice/fix-ingress-share-real

    Revert share ingress to dedicated hostname + enable redirect to /share

commit ca20a8c
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 13:15:37 2024 +0100

    redirect to /share

commit a682b79
Merge: 066cecc 1780e23
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 11:23:05 2024 +0100

    Merge pull request #74 from ministryofjustice/remove-share-sub-domain

    Remove share subdomain

commit 1780e23
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 11:21:18 2024 +0100

    remove share subdomain

commit 066cecc
Merge: 4d23e4b b21f656
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:36:49 2024 +0100

    Merge pull request #73 from ministryofjustice/workflow-changes

    Update cloud-platform-deploy-release.yml

commit b21f656
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:35:57 2024 +0100

    Update cloud-platform-deploy-release.yml

commit 4d23e4b
Merge: 6aeb967 ca1cfb2
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:09:57 2024 +0100

    Merge pull request #72 from ministryofjustice/add-airflow-ips

    chore: Add templated directory to .gitignore and update IP whitelist in values files

commit ca1cfb2
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:08:14 2024 +0100

    chore: Add templated directory to .gitignore and update IP whitelist in values files

commit 6aeb967
Merge: f6a7c0e 540fe64
Author: Prem Basumatary <[email protected]>
Date:   Fri Jul 5 09:10:27 2024 +0100

    Merge pull request #71 from ministryofjustice/NIT-1305-custom-scaling-times

    NIT-1305 fix error in script

commit 540fe64
Author: Prem Basumatary <[email protected]>
Date:   Fri Jul 5 09:03:12 2024 +0100

    NIT-1305 fix error in script

* Update ingress-share.yaml

* vals

* kustomize base

* ingress

* setup makefiel

* ingress poc patches

* correct poc ingress

* Update values.yaml

* update for helm values mergign

* updates

* kustomization base

* rearrange patching for ingress to repeat less

* fix ingress patching with allow list

* :fire: remove alf charts

* rename values files

* :fire: disable reindexing job from helm values

* :sparkles: Uncouple the reindexing job from the main helm charts

tidy

* Update values.yaml

* Update values.yaml

* ♻️  Get rds url from secret rather than hard coding (#85)

* update values for opensearch

* add makefile +chart changes

* Update values.yaml

* Update values.yaml

* update makefile

* Update values.yaml

* Update values.yaml

* Update values.yaml

* vars

* Update values.yaml

* Update values.yaml

* images

* config map

* Revert "config map"

This reverts commit e714ada.

* Reapply "config map"

This reverts commit c47acb8.

* config map

* images

* Squashed commit of the following:

commit 6cda4f5
Merge: a682b79 ca20a8c
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 15:03:01 2024 +0100

    Merge pull request #75 from ministryofjustice/fix-ingress-share-real

    Revert share ingress to dedicated hostname + enable redirect to /share

commit ca20a8c
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 13:15:37 2024 +0100

    redirect to /share

commit a682b79
Merge: 066cecc 1780e23
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 11:23:05 2024 +0100

    Merge pull request #74 from ministryofjustice/remove-share-sub-domain

    Remove share subdomain

commit 1780e23
Author: George Taylor <[email protected]>
Date:   Tue Jul 16 11:21:18 2024 +0100

    remove share subdomain

commit 066cecc
Merge: 4d23e4b b21f656
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:36:49 2024 +0100

    Merge pull request #73 from ministryofjustice/workflow-changes

    Update cloud-platform-deploy-release.yml

commit b21f656
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:35:57 2024 +0100

    Update cloud-platform-deploy-release.yml

commit 4d23e4b
Merge: 6aeb967 ca1cfb2
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:09:57 2024 +0100

    Merge pull request #72 from ministryofjustice/add-airflow-ips

    chore: Add templated directory to .gitignore and update IP whitelist in values files

commit ca1cfb2
Author: George Taylor <[email protected]>
Date:   Thu Jul 11 15:08:14 2024 +0100

    chore: Add templated directory to .gitignore and update IP whitelist in values files

commit 6aeb967
Merge: f6a7c0e 540fe64
Author: Prem Basumatary <[email protected]>
Date:   Fri Jul 5 09:10:27 2024 +0100

    Merge pull request #71 from ministryofjustice/NIT-1305-custom-scaling-times

    NIT-1305 fix error in script

commit 540fe64
Author: Prem Basumatary <[email protected]>
Date:   Fri Jul 5 09:03:12 2024 +0100

    NIT-1305 fix error in script

* Update ingress-share.yaml

* vals

* kustomize base

* ingress

* setup makefiel

* ingress poc patches

* correct poc ingress

* Update values.yaml

* update for helm values mergign

* updates

* kustomization base

* rearrange patching for ingress to repeat less

* fix ingress patching with allow list

* :fire: remove alf charts

* rename values files

* :fire: disable reindexing job from helm values

* :sparkles: Uncouple the reindexing job from the main helm charts

tidy

* set helm values dynamically

* Update makefile

* :wrench: changes from dev deploy

* :wrench: Add test environment

* ✏️ incorrect ref in naming

* amq

---------

Co-authored-by: pete-j-g <[email protected]>
Co-authored-by: Piotr Grzeskowiak <[email protected]>
Co-authored-by: Prem Basumatary <[email protected]>
Co-authored-by: Shahzad Chaudhry <[email protected]>
Co-authored-by: Shahzad Chaudhry <[email protected]>
Co-authored-by: Prem Basumatary <[email protected]>
Co-authored-by: Andrew Moore <[email protected]>
Co-authored-by: Andrew Moore <[email protected]>
  • Loading branch information
9 people authored Aug 21, 2024
1 parent b6695ea commit 1edfdb8
Show file tree
Hide file tree
Showing 20 changed files with 200 additions and 51 deletions.
3 changes: 1 addition & 2 deletions .github/workflows/scale-down.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ on:
schedule:
- cron: '00 19 * * 1-5'
workflow_dispatch:

jobs:
scale-down:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -51,4 +50,4 @@ jobs:
for deployment in $DEPLOYMENTS; do
kubectl scale $deployment --replicas=0
done
done
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@
.idea/
templated/
kustomize/base/charts/
kustomize/overlays/**/charts/
kustomize/overlays/**/output.yaml
kustomize/**/charts/
kustomize/**/output.yaml
kustomize/base/resources.yaml
3 changes: 0 additions & 3 deletions jobs/reindex/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,5 @@
{{- end -}}

{{- define "alfresco-search-enterprise.config.spring" -}}
{{- if and (not .Values.global.elasticsearch.host) (not .Values.searchIndex.host) }}
{{ fail "Please provide external elasticsearch connection details as values under .global.elasticsearch or .searchIndex or enable the embedded elasticsearch via .elasticsearch.enabled" }}
{{- end }}
SPRING_ELASTICSEARCH_REST_URIS: "{{ .Values.global.elasticsearch.protocol }}://{{ .Values.global.elasticsearch.host }}:{{ .Values.global.elasticsearch.port }}"
{{- end -}}
6 changes: 3 additions & 3 deletions jobs/reindex/templates/reindexing-config.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: delius-alfresco-search-enterprise-reindexing-configmap
name: alfresco-content-services-alfresco-search-enterprise-reindexing-configmap
data:
ELASTICSEARCH_INDEXNAME: "{{ .Values.indexName }}"
{{ template "alfresco-search-enterprise.config.spring" . }}
ALFRESCO_SHAREDFILESTORE_BASEURL: http://delius-alfresco-filestore:80/alfresco/api/-default-/private/sfs/versions/1/file/
ALFRESCO_ACCEPTEDCONTENTMEDIATYPESCACHE_BASEURL: http://delius-alfresco-router/transform/config
ALFRESCO_SHAREDFILESTORE_BASEURL: http://alfresco-content-services-alfresco-filestore:80/alfresco/api/-default-/private/sfs/versions/1/file/
ALFRESCO_ACCEPTEDCONTENTMEDIATYPESCACHE_BASEURL: http://alfresco-content-services-alfresco-router/transform/config
ALFRESCO_REINDEX_PATHINDEXINGENABLED: {{ .Values.pathIndexingEnabled | quote }}
SPRING_DATASOURCE_URL: {{ .Values.postgresql.url }}
{{- if .Values.environment }}
Expand Down
14 changes: 7 additions & 7 deletions jobs/reindex/templates/reindexing-job.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: batch/v1
kind: Job
metadata:
name: delius-alfresco-search-enterprise-reindexing
name: alfresco-content-services-alfresco-search-enterprise-reindexing
spec:
template:
metadata:
Expand All @@ -23,7 +23,7 @@ spec:
- name: {{ .Values.imagePullSecrets }}
restartPolicy: Never
containers:
- name: delius-alfresco-search-enterprise-reindexing
- name: alfresco-content-services-alfresco-search-enterprise-reindexing
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
securityContext:
Expand All @@ -36,12 +36,12 @@ spec:
type: RuntimeDefault
resources: {{- toYaml .Values.resources | nindent 12 }}
volumeMounts:
- name: delius-alfresco-reindexing-prefixes-file-volume
- name: reindexing-prefixes-file-volume
mountPath: /alf/reindex.prefixes-file.json
subPath: reindex.prefixes-file.json
envFrom:
- configMapRef:
name: delius-alfresco-search-enterprise-reindexing-configmap
name: alfresco-content-services-alfresco-search-enterprise-reindexing-configmap
- secretRef:
name: {{ .Values.messageBroker.existingSecretName }} # Ensure this value is set correctly
env:
Expand All @@ -68,9 +68,9 @@ spec:
containerPort: 8080
protocol: TCP
volumes:
- name: delius-alfresco-reindexing-prefixes-file-volume
- name: reindexing-prefixes-file-volume
configMap:
name: delius-alfresco-reindexing-prefixes-file-configmap
name: reindexing-prefixes-file-configmap
initContainers:
- name: wait-for-repository
image: curlimages/curl:7.79.1
Expand All @@ -83,7 +83,7 @@ spec:
resources: {{- toYaml .Values.initcontainers.waitForRepository.resources | nindent 12 }}
env:
- name: ALFRESCO_REPOSITORY_URL
value: http://delius-alfresco-cs-repository/alfresco/api/-default-/public/alfresco/versions/1/probes/-ready-
value: http://alfresco-content-services-alfresco-cs-repository/alfresco/api/-default-/public/alfresco/versions/1/probes/-ready-
command: [ "/bin/sh","-c" ]
# Delay running the reindexing to give Alfresco Repository a chance to fully initialise
args: [ "while [ $(curl -sw '%{http_code}' $ALFRESCO_REPOSITORY_URL -o /dev/null) -ne 200 ]; do sleep 5; echo 'Waiting for the Alfresco Repository...'; done; echo 'Alfresco is ready, delay reindexing to give a chance to fully initialise.'; sleep 30; echo 'Reindexing started!'" ]
2 changes: 1 addition & 1 deletion jobs/reindex/templates/reindexing-prefixes-config-map.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: delius-alfresco-reindexing-prefixes-file-configmap
name: reindexing-prefixes-file-configmap
data:
reindex.prefixes-file.json: |
{
Expand Down
2 changes: 1 addition & 1 deletion kustomize/base/patch-delete-reindexing-config-map.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ $patch: delete
apiVersion: v1
kind: ConfigMap
metadata:
name: delius-alfresco-search-enterprise-reindexing-configmap
name: alfresco-content-services-alfresco-search-enterprise-reindexing-configmap
2 changes: 1 addition & 1 deletion kustomize/base/patch-ingress-repository.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: delius-alfresco-cs-repository
name: alfresco-content-services-alfresco-cs-repository
annotations:
external-dns.alpha.kubernetes.io/aws-weight: "100"
kubernetes.io/ingress.class: default
Expand Down
6 changes: 5 additions & 1 deletion kustomize/base/patch-ingress-share.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: delius-alfresco-cs-share
name: alfresco-content-services-alfresco-cs-share
annotations:
external-dns.alpha.kubernetes.io/aws-weight: "100"
kubernetes.io/ingress.class: default
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($request_uri = "/") {
return 301 /share/page/;
}
spec:
ingressClassName: default
13 changes: 10 additions & 3 deletions kustomize/base/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ repository:
type: Recreate
image:
repository: 754256621582.dkr.ecr.eu-west-2.amazonaws.com/hmpps-migration/alfresco-content-repository
tag: content-9895398808-1
tag: release_7.3.2_elasticsearch-r5.0-content-latest
pullPolicy: IfNotPresent
internalPort: 8080
hazelcastPort: 5701
Expand Down Expand Up @@ -639,7 +639,7 @@ share:
image:
# repository: quay.io/alfresco/alfresco-share
repository: 754256621582.dkr.ecr.eu-west-2.amazonaws.com/hmpps-migration/alfresco-share
tag: share-9895398808-1
tag: release_7.3.2_elasticsearch-r5.0-share-10369065562-1
pullPolicy: IfNotPresent
internalPort: 8080
service:
Expand Down Expand Up @@ -691,6 +691,13 @@ share:
activemq:
nameOverride: activemq
enabled: true
resources:
requests:
cpu: "0.5"
memory: "1024Mi"
limits:
cpu: "2"
memory: "4096Mi"
# -- Possibility to choose Node for pod, with a key-value pair label
# e.g {"kubernetes.io/hostname": multinode-demo-m02}
nodeSelector: {}
Expand Down Expand Up @@ -781,7 +788,7 @@ alfresco-search-enterprise:
existingSecretName: *acs_messageBroker_secretName
searchIndex:
# -- The host where service is available
host: opensearch-proxy-service-cloud-platform-62a206e0.hmpps-delius-alfrsco-poc.svc.cluster.local
host: null
# -- The port where service is available
port: 8080
# -- Valid values are http or https
Expand Down
10 changes: 5 additions & 5 deletions kustomize/dev/patch-ingress-repository.yaml
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: delius-alfresco-cs-repository
name: alfresco-content-services-alfresco-cs-repository
annotations:
external-dns.alpha.kubernetes.io/set-identifier: delius-alfresco-cs-repository-hmpps-delius-alfresco-dev-green
nginx.ingress.kubernetes.io/whitelist-source-range: placeholder
external-dns.alpha.kubernetes.io/set-identifier: alfresco-content-services-alfresco-cs-repository-hmpps-delius-alfresco-dev-green
nginx.ingress.kubernetes.io/whitelist-source-range: "placeholder"
spec:
rules:
- host: hmpps-delius-alfresco-dev.apps.live.cloud-platform.service.justice.gov.uk
http:
paths:
- backend:
service:
name: delius-alfresco-cs-repository
name: alfresco-content-services-alfresco-cs-repository
port:
number: 80
path: /
pathType: Prefix
- backend:
service:
name: delius-alfresco-cs-repository
name: alfresco-content-services-alfresco-cs-repository
port:
number: 80
path: /api-explorer
Expand Down
14 changes: 7 additions & 7 deletions kustomize/dev/patch-ingress-share.yaml
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: delius-alfresco-cs-share
name: alfresco-content-services-alfresco-cs-share
annotations:
external-dns.alpha.kubernetes.io/set-identifier: delius-alfresco-cs-share-hmpps-delius-alfrsco-poc-green
nginx.ingress.kubernetes.io/whitelist-source-range: placeholder
external-dns.alpha.kubernetes.io/set-identifier: alfresco-content-services-alfresco-cs-share-hmpps-delius-alfresco-dev-green
nginx.ingress.kubernetes.io/whitelist-source-range: "placeholder"
spec:
rules:
- host: share.hmpps-delius-alfrsco-poc.apps.live.cloud-platform.service.justice.gov.uk
- host: share.hmpps-delius-alfresco-dev.apps.live.cloud-platform.service.justice.gov.uk
http:
paths:
- backend:
service:
name: delius-alfresco-cs-share
name: alfresco-content-services-alfresco-cs-share
port:
number: 80
path: /
pathType: Prefix
- backend:
service:
name: delius-alfresco-cs-share
name: alfresco-content-services-alfresco-cs-share
port:
number: 80
path: /share/page/
pathType: Prefix
tls:
- hosts:
- share.hmpps-delius-alfrsco-poc.apps.live.cloud-platform.service.justice.gov.uk
- share.hmpps-delius-alfresco-dev.apps.live.cloud-platform.service.justice.gov.uk
secretName: share-ingress-cert
9 changes: 4 additions & 5 deletions kustomize/poc/patch-ingress-repository.yaml
Original file line number Diff line number Diff line change
@@ -1,26 +1,25 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: delius-alfresco-cs-repository
name: alfresco-content-services-alfresco-cs-repository
annotations:
external-dns.alpha.kubernetes.io/set-identifier: delius-alfresco-cs-repository-hmpps-delius-alfrsco-poc-green
external-dns.alpha.kubernetes.io/set-identifier: alfresco-content-services-alfresco-cs-repository-hmpps-delius-alfrsco-poc-green
nginx.ingress.kubernetes.io/whitelist-source-range: "placeholder"
nginx.ingress.kubernetes.io/whitelist-source-range: placeholder
spec:
rules:
- host: hmpps-delius-alfrsco-poc.apps.live.cloud-platform.service.justice.gov.uk
http:
paths:
- backend:
service:
name: delius-alfresco-cs-repository
name: alfresco-content-services-alfresco-cs-repository
port:
number: 80
path: /
pathType: Prefix
- backend:
service:
name: delius-alfresco-cs-repository
name: alfresco-content-services-alfresco-cs-repository
port:
number: 80
path: /api-explorer
Expand Down
8 changes: 4 additions & 4 deletions kustomize/poc/patch-ingress-share.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: delius-alfresco-cs-share
name: alfresco-content-services-alfresco-cs-share
annotations:
external-dns.alpha.kubernetes.io/set-identifier: delius-alfresco-cs-share-hmpps-delius-alfrsco-poc-green
external-dns.alpha.kubernetes.io/set-identifier: alfresco-content-services-alfresco-cs-share-hmpps-delius-alfrsco-poc-green
nginx.ingress.kubernetes.io/whitelist-source-range: placeholder
spec:
rules:
Expand All @@ -12,14 +12,14 @@ spec:
paths:
- backend:
service:
name: delius-alfresco-cs-share
name: alfresco-content-services-alfresco-cs-share
port:
number: 80
path: /
pathType: Prefix
- backend:
service:
name: delius-alfresco-cs-share
name: alfresco-content-services-alfresco-cs-share
port:
number: 80
path: /share/page/
Expand Down
46 changes: 46 additions & 0 deletions kustomize/test/allowlist.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
- "35.176.126.163" # legacy delius-test-az1-nat-gateway
- "35.178.162.73" # legacy delius-test-az2-nat-gateway
- "52.56.195.113" # legacy delius-test-az3-nat-gateway
- "35.178.209.113" # Cloud Platform live-1-eu-west-2a
- "3.8.51.207" # Cloud Platform live-1-eu-west-2c
- "35.177.252.54" # Cloud Platform live-1-eu-west-2b
- "35.176.93.186/32" # MoJ GlobalProtect
- "35.177.125.252/32" # MoJ VPN Gateway Proxies
- "35.177.137.160/32" # MoJ VPN Gateway Proxies
- "81.134.202.29/32" # MoJ VPN
- "51.149.250.0/24" # PTTP / MoJO Production Account BYOIP CIDR range
- "51.149.251.0/24" # PTTP / MoJO Production Account BYOIP CIDR range - PreProd
- "213.121.161.112/28" # 102 Petty France WiFi
- "217.33.148.210/32" # Digital studio
- "13.43.9.198/32" # MP non_live_data-public-eu-west-2a-nat
- "13.42.163.245/32" # MP non_live_data-public-eu-west-2b-nat
- "18.132.208.127/32" # MP non_live_data-public-eu-west-2c-nat
- "51.149.249.0/29" # ARK Corsham Internet Egress Exponential-E
- "51.149.249.32/29" # ARK Corsham Internet Egress Exponential-E
- "194.33.192.0/25" # ARK internet (DOM1)
- "194.33.193.0/25" # ARK internet (DOM1)
- "194.33.196.0/25" # ARK internet (DOM1)
- "194.33.197.0/25" # ARK internet (DOM1)
- "195.59.75.0/24" # ARK internet (DOM1)
- "194.33.248.0/29" # ARK Corsham Internet Egress Vodafone
- "194.33.249.0/29" # ARK Corsham Internet Egress Vodafone
- "62.25.106.209/32" # OMNI
- "195.92.40.49/32" # OMNI
- "62.25.109.197/32" # Quantum
- "195.92.38.16/28" # Quantum
- "212.137.36.230/32" # Quantum
- "78.33.10.50/31" # Unilink AOVPN
- "78.33.10.52/30" # Unilink AOVPN
- "78.33.10.56/30" # Unilink AOVPN
- "78.33.10.60/32" # Unilink AOVPN
- "78.33.32.99/32" # Unilink AOVPN
- "78.33.32.100/30" # Unilink AOVPN
- "78.33.32.104/30" # Unilink AOVPN
- "78.33.32.108/32" # Unilink AOVPN
- "83.98.63.176/29" # Unilink AOVPN
- "194.75.210.216/29" # Unilink AOVPN
- "217.138.45.109/32" # Unilink AOVPN
- "217.138.45.110/32" # Unilink AOVPN
- "34.241.149.106/32" # Airflow in Analytical Platform
- "52.210.79.20/32" # Airflow in Analytical Platform
- "54.228.134.38/32" # Airflow in Analytical Platform
9 changes: 9 additions & 0 deletions kustomize/test/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../base

patches:
- path: patch-ingress-repository.yaml
- path: patch-ingress-share.yaml
29 changes: 29 additions & 0 deletions kustomize/test/patch-ingress-repository.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: alfresco-content-services-alfresco-cs-repository
annotations:
external-dns.alpha.kubernetes.io/set-identifier: alfresco-content-services-alfresco-cs-repository-hmpps-delius-alfresco-test-green
nginx.ingress.kubernetes.io/whitelist-source-range: "placeholder"
spec:
rules:
- host: hmpps-delius-alfresco-test.apps.live.cloud-platform.service.justice.gov.uk
http:
paths:
- backend:
service:
name: alfresco-content-services-alfresco-cs-repository
port:
number: 80
path: /
pathType: Prefix
- backend:
service:
name: alfresco-content-services-alfresco-cs-repository
port:
number: 80
path: /api-explorer
pathType: Prefix
tls:
- hosts:
- hmpps-delius-alfresco-test.apps.live.cloud-platform.service.justice.gov.uk
Loading

0 comments on commit 1edfdb8

Please sign in to comment.