Move "pacts for verification" configuration into the Pact Broker

[bethesque]

Background

Currently, the configuration for which pacts to verify lives in the provider test code. eg.

const verificationOptions = {
  consumerVersionSelectors: [
    {
      tag: "main",
      latest: true
    },
    {
      tag: process.env.GIT_BRANCH,
      latest: true
    },
    {
      tag: "test",
      latest: true
    },
    {
      tag: "production",
      latest: true
    }
  ],
  enablePending: true,
  includeWipPactsSince: process.env.GIT_BRANCH === "main" ? "2020-01-01" : undefined,
}

When branches and environments/deployments/releases support is out, the need for each provider project to be individually configured will mostly go away. The pacts that should be verified will be { mainBranch: true }, { deployedOrReleased: true }. If no pacts are specified in the consumer version selectors, these will now the the pacts returned, meaning that users do not need to understand selectors to get started with Pact.

The only thing that is likely to change between companies is how teams coordinate the development of features between the consumer and provider. Some teams use matching branch names so that the feat/x pact gets implemented on the feat/x branch. Other teams have branch names specified for them via a development process (eg. feat/jira-1234), and so need to use another method of selecting the relevant pact(s) for a provider feature branch. This can result in the consumer version selectors being modified and committed on a feature branch, and then reset back to base after being merged.

Proposal

Move the 'pacts for verification' configuration into the Broker and out of the provider code.

Why?

• Makes it simpler to get started - no need to understand what to configure.
• The best practice configuration is used by default. Conscious action must be taken to alter the configuration.
• Adding new selectors and options won't require updates to 10+ client libraries each time. (eg. on the backlog are branches support, regular expressions support for tags, more ways to customise the WIP pacts, support for ignoring pacts etc)
• Putting the logic of how to map the consumer feature branches to provider feature branches (eg. by matching tags, branches, or regex) will allow the contract_requiring_verification webhook to target the correct feature branch when the pact content changes on a consumer feature branch (currently it can only target the main branch and the deployed/released versions).
• If a pattern can be used to determine how to map consumer branches to provider branches, the selectors would not need to be updated on the provider feature branch and then reset each time.

Cons

• The actual configuration is less obvious to the end user, and couldn't be changed without write access to the Broker. This may be able to be mitigated by adding more detail to the 'pacts for verification' response.
• The broker side configuration won't be able to be programmatically generated on per-branch basis (though all the existing client side configurations will still work)

Considerations

• If the configuration lives in the Broker, the logic of which pacts to verify could not be custom coded per branch - we'd need to provide a way to specify things like "only include WIP pacts on the main branch".
• The 'pacts for verification' API would need to work with existing client side configuration as well as the new broker side configuration.
• If there was an unpredictable mapping between consumer/provider branch names, the developer would need a way to specify the feature pact(s) to verify without requiring write access to the Broker, and without having to duplicate the entire configuration client side just to add one pact.

Proposed design

• When the 'pacts for verification' endpoint is called:
  • If no consumer version selectors are provided, the provider's server side selectors will be used. If the provider has no server side selectors, the default selectors { mainBranch: true }, { deployedOrReleased: true }, { matchingBranch: true }, { matchingTag: true } will be used.
  • If enablePending is not specified, it will default to true.
  • If no includeWipPactsSince is specified, WIP pacts will be returned for the main branch.

Configuration endpoint

PUT /pacticipants/my-provider/verification-configuration
{
  "consumerVersionSelectors": [...],
  "enablePending": true,
  "includeWipPactsSince": "2021-01-01",
  "includeWipPactsFor": [{ mainBranch: true }]
}

Configuration CLI

pact-broker configure-verification --pacticipant my-provider \
                      --consumer-version-selector "{...}" \
                      --consumer-version-selector "{...}" \
                      --enable-pending \
                      --include-wip-pacts-since "2021-01-01"

[solarmosaic-kflorence]

What will the default value of includeWipPactsSince be? A year ago? Would be nice to eventually allow matchingBranch to accept a regex. We have to use that here due to all of our services being in a monorepo.

[bethesque]

I think the "since" part of it is only really relevant for teams that already have a back log of pacts in their broker. It won't be necessary for teams starting out new, so it could really be any arbitrary date in the past - a year seems fine. I'd like to support better selectors for WIP, or add { ignore: ... } selectors, because the date seems a bit "icky".

Here's the feature request for the tag regex. Please add any extra use cases. https://pact.canny.io/feature-requests/p/allow-version-selectors-to-specify-tags-with-regex

[solarmosaic-kflorence]

Yes, it is icky :)

[bethesque]

It was a tactical solution to help people start using the WIP feature without having to go through and clean up all the old data manually. Someone has picked up the work to allow people to delete pacts by tag though the UI, which will make the clean up easier.

[pendsley]

Love it. Moving the intricacies takes this burden off the provider teams. When the provider specifies selectors, it would almost always lead into questions about what they were doing and why, which would lead into nitty gritty details of how pact was working behind the scenes. Good stuff to understand, but it's overwhelming for new adopters/teams just dipping their toes in.

If a pattern can be used to determine how to map consumer branches to provider branches, the selectors would not need to be updated on the provider feature branch and then reset each time

I might have missed it, but how will this be addressed with the proposal.

[bethesque]

I had a couple of thoughts, which I didn't actually write down, well spotted. One was that there could be a command that gets run by the provider developer that puts the mapping directly in the broker.

eg.

pact-broker update-verification-configuration --pacticipant Foo --custom-branch-mapping <consumer_branch> <provider_branch>

The advantage of this is that if the consumer then went and updated their pact, the Broker would know to trigger a build of the mapped provider branch in the contract_requiring_verification_published webhook (which is discussed here).

The disadvantage of this is that the developer would need write credentials, and not every company gives those out to the developers for use from their local machines.

The other thought was making that regular expression matching more powerful and do something smarter with tags. Because branches will have proper support soon, we won't need to use tags to represent branches any more, and they could be used to signify features instead. Then we could support a direct or regex based mapping between tags.

[bethesque]

Updated thoughts on this configuration. When doing the configuration in the client, different configurations can be set dynamically based on the information that the client has (eg. branch). The serverside configuration would need to be able to support this also. Proposing a set of "rules" rather than a single configuration.

Same configuration for every branch. Can be configured via command line.

pact-broker configure-verification --pacticipant my-provider \
                      --consumer-version-selector '{"mainBranch": true}' \
                      --consumer-version-selector '{"deployedAndReleased": true}' \
                      --enable-pending \
                      --include-wip-pacts-since "2021-01-01"

{
  "rules": [
    {
      "consumerVersionSelectors": [{ "mainBranch": true }, { "deployedAndReleased": true }],
      "enablePending": false,
      "includeWipPactsSince": "2020-01-01"
    }
  ]
}

Different configuration for feature branches - allow a provider version matcher to be specified that
allows a different configuration to be applied based on the branch (or tag). Would be too complex to configure via command line,
so a config file can be provided.

Rules checked in order of declaration until a matching one is found.

{
  "rules": [
    {
      "description": "Main branch configuration",
      "providerVersionMatchers": [ { "mainBranch": true } ],
      "consumerVersionSelectors": [ { "mainBranch": true }, { "deployedAndReleased": true } ],
      "enablePending": true,
      "includeWipPactsSince": "2020-01-01"
    },
    {
      "description": "Feature branch configuration (anything not main branch)",
      "consumerVersionSelectors": [ { "mainBranch": true }, { "matchingBranch": true }, { "deployedAndReleased": true } ],
      "enablePending": false,
      "includeWipPactsSince": "2020-01-01"
    }
  ]
}

pact-broker configure-verification --pacticipant my-provider --file "myconfig.json"

[praveen-em]

love this feature!