Skip to content

Packaging Process

Jump to bottom Edit New page
Gürkan İndibay edited this page Oct 4, 2021 · 29 revisions

Packaging process consists of several steps

  1. Updates version information in packaging files and opens a PR to be reviewed: At this step, packaging configuration files packaging type specific files (i.e. deb and rpm) are updated with the new version information

    Files and the changes performed in the files are as below

    • Packaging configuration files
      • pkgvars: Version information is changed in this file.
    • Packaging type specific files
      • citus.spec: Version information in Version, Source0 is changed and a new changelog entry is added into changelog field
      • debian/changelog: New changelog entries are added at the top of the file To execute and complete this step following pipeline should be executed with appropriate parameters

    Pipeline to be executed is as below

    https://github.com/citusdata/packaging/actions/workflows/update_package_properties.yml

    When pipeline is opened, you can see the "Run Workflow" button at the top left of the executions list as below

    First branch should be chosen for the parameters to appear

    image

    Available branches and their corresponding project to be chosen as below

    Application Branch
    citus all-citus
    citus-enterprise all-enterprise
    pg-auto-failover all-pgautofailover
    pg-auto-failover-enterprise all-pgautofailover-enterprise

  2. Build and publish packages into packagecloud repo This is an automatically triggered pipeline when the pipeline in the first step is executed. This pipeline is executed twice. Once PR is opened and once PR is reviewed and merged. Before merge, only packages are built but they aren't pushed. After PR close so merge operation, packages is built again bu this time they are pushed.

  3. Build and publish packages into Microsoft Packages This step is only available for enterprise applications(i.e. citus-enterprise and pg-auto-failover-enterprise). The pipeline and the packaging code for this step are both hosted on Azure Devops. Since there is no connection with our main repository,i.e GitHub, we need to trigger this pipeline manually. Sources are stored in the below Azure Devops Repo

https://msdata.visualstudio.com/Database%20Systems/_git/citus-packaging

There are two branches that stores our packaging codes

  1. 'all-enterprise': Stores citus-enterprise packaging codes
  2. 'all-pg-auto-failover-enterprise': Stores all-pg-auto-failover-enterprise codes There is one more branch in this repository 'repoclient' , which stores Microsoft Packages client, ehich enables us pushing and listing packages and listing repositories Pipeline address is as below https://msdata.visualstudio.com/Database%20Systems/_build?definitionId=10018&_a=summary

This pipeline is automatically executed when code is pushed into 'all-enterprise' and 'all-pg-auto-failover-enterprise'. If succeeefully completed, packages will be pushed into Microsoft

  1. Update version information for docker images Docker image files include citus version information to define the citus version to be downloaded. A docker file exists for each postgres version. This step automatically updates the citus version information in these docker files. There are four docker files that is used to publish images and one docker-compose file to create a multi-node citus stack .However, first four are only updated. There is no need to update the nightly since it is getting the latest citus.

https://github.com/citusdata/docker/actions/workflows/update_version.yml

image

  1. Build and publish docker images After completion of step 4; i.e. created PR approved and merged; "Publish docker image" is triggered and this pipeline builds and publishes the docker images mentioned in step 4.

When publishin these images, tags for the related version are created as well for main, alpine and postgres-12 images

Tags created for each images defined above is as below: Suppose that version is as X.Y.Z. Following tags should be created and pushed into DockerHub

  • citusdata/citus:X
  • citusdata/citus:X.Y
  • citusdata/citus:X.Y.Z
  • citusdata/citus:X-alpine
  • citusdata/citus:X.Y-alpine
  • citusdata/citus:X.Y.Z-alpine
  • citusdata/citus:X-pg12
  • citusdata/citus:X.Y-pg12
  • citusdata/citus:X.Y.Z-pg12

After completion of this pipeline, images should be checked on dockerhub in the address below https://hub.docker.com/repository/registry-1.docker.io/citusdata/citus/tags?page=1&ordering=last_updated

  1. Update version information for pgxn This pipeline updates the version information on PGXN files below
  • META.json
  • pkgvars

After completion of pipeline, a PR is created for changes to be checked.

  1. Pgxn Publish pipeline. After merging the changes performed in step 6, Pgxn publish pipeline is triggered automatically. Results can be checked in the below address, https://github.com/citusdata/packaging/actions/workflows/build_package.yml After successful completion of this release, package should be checked whether it is updated on pgxn site using the address below https://pgxn.org/dist/citus/

  2. Create an issue for pgdg repositories

After completion of all the steps above, lastly an issue should be opened in postgresql redmine site from https://redmine.postgresql.org/projects/pgrpms/issues/new

A sample issue is as below. https://redmine.postgresql.org/issues/6693

Some steps is not executed for some applications. Steps to be applied according to the applications

Application Applied steps
citus 1,2,4,5,6,7,8
citus-enterprise 1,2,3
pg-auto-failover 1,2
pg-auto-failover-enterprise 1,2,3
Add a custom footer
Add a custom sidebar
Clone this wiki locally