From eec44e3e903525cf5fb0ee7af9761e1c94ab8367 Mon Sep 17 00:00:00 2001 From: kodjomiles Date: Tue, 10 Dec 2024 13:46:06 +0000 Subject: [PATCH 1/6] adding additional tracking methods --- docs/guides/all/setup-dora-metrics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/guides/all/setup-dora-metrics.md b/docs/guides/all/setup-dora-metrics.md index 6c040f42c..48757f7ef 100644 --- a/docs/guides/all/setup-dora-metrics.md +++ b/docs/guides/all/setup-dora-metrics.md @@ -410,7 +410,6 @@ By following a similar pattern for other tools, you will be able to capture depl - CI/CD pipelines, such as those run by Jenkins, provide a robust way to track deployments. Jenkins, in particular, allows you to create and update entities in Port dynamically using Port's API as part of the pipeline execution. @@ -793,7 +792,6 @@ Unlike GitHub, **GitLab** and **Azure DevOps** do not support traditional releas - If your tool or workflow is not natively supported, you can create custom integrations by directly interacting with Port’s API. This method allows you to track deployments from any system that can make HTTP API calls. @@ -848,6 +846,8 @@ giving you full flexibility across all your deployment workflows.
+ + ### Monorepo Tracking By using **custom integrations**, you can effectively track services or components within a monorepo. From 2b00a5d6933450fa66459045c21e9ae6337c271c Mon Sep 17 00:00:00 2001 From: kodjomiles Date: Wed, 11 Dec 2024 12:46:57 +0000 Subject: [PATCH 2/6] Added additional ways to track deployment --- docs/guides/all/setup-dora-metrics.md | 187 +++++++++++++++++++++++++- 1 file changed, 183 insertions(+), 4 deletions(-) diff --git a/docs/guides/all/setup-dora-metrics.md b/docs/guides/all/setup-dora-metrics.md index 48757f7ef..e83959919 100644 --- a/docs/guides/all/setup-dora-metrics.md +++ b/docs/guides/all/setup-dora-metrics.md @@ -24,6 +24,7 @@ This guide will cover the four key metrics: **deployment frequency**, **lead tim - Access to a repository (GitHub, GitLab, or Azure Repos) that is connected to Port via the onboarding process. - While this guide demonstrates implementations using **GitHub**, **GitLab**, and **Azure Repos**, other Git providers can be used as well. - Optional for advanced strategies: If you're using workflows or pipelines, ensure they are configured for deployment tracking by following the relevant setup guides, such as CI/CD integrations or your platform-specific tools. +- Optional for alternate tracking strategies: Install the [Port Jira integration](/build-your-software-catalog/sync-data-to-catalog/project-management/jira) to track deployments, lead time and change failure rate using Jira. ## Tracking deployments @@ -130,6 +131,13 @@ By following these steps, you can paste and manage the JSON schema required to t ### Tracking strategies Below are the main ways you can track deployments directly within Port: + + + + -
+ + + + +Although PR merge is the recommended way of tracking deployments, besides the CI methods described above, +you can also track and measure DORA metrics using other approaches that may align better with your team's workflow. + + + + + +Instead of relying solely on PR merges or pipeline runs, you can treat a Jira ticket moving to the "Done" status as an indicator that code has effectively been deployed. + +
+Track deployments based on Jira ticket status (click to expand) +```yaml showLineNumbers +- kind: issue + selector: + # Use a JQL or a query that filters for issues with status = Done + jql: 'status = Done' + port: + entity: + mappings: + identifier: .key + title: .fields.summary + blueprint: '"deployment"' + properties: + # Using the issue's creation time as the deployment time + createdAt: .fields.created + # Marking all completed issues as successful deployments + deploymentStatus: '"Success"' +``` + +
+ + + + + +Calculate lead time based on how long it takes a Jira ticket to move from “In Progress” to “Done.” +This approach focuses on a more business-oriented measure of lead time, tying it directly to ticket lifecycle rather than just code merges. + +
+Calculate lead time on Jira ticket status (click to expand) + +```yaml showLineNumbers +
+ +```yaml showLineNumbers +- kind: issue + selector: + jql: 'status in ("In Progress", "Done")' + port: + entity: + mappings: + identifier: .key + title: .fields.summary + blueprint: '"jiraIssue"' + properties: + # Calculate lead time in hours from created to resolutionDate + leadTimeHours: > + (.fields.created as $created | .fields.resolutiondate as $resolved | + if $resolved == null then null else + ((($resolved | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) - + ($created | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime)) / 3600) end) +``` + + + + + + + +Count the number of urgent Jira bugs that appear after deployments to gauge the change failure rate. +This method aligns failure rate with customer-impacting issues tracked in your ticketing system. + +To implement this follow the steps below: + +1. Add an aggregation property to the Jira `project` blueprint to count high-priority issues on a project: + +
+Aggregation property for high-priority issues (click to expand) + +```json showLineNumbers +"highestPriorityIssueCount": { +"title": "Highest Priority Issue Count", +"target": "jiraIssue", +"calculationSpec": { + "calculationBy": "entities", + "func": "count" +}, +"query": { + "combinator": "and", + "rules": [ + { + "property": "priority", + "operator": "=", + "value": "Highest" + } + ] +} +} + +``` +
+ + :::tip Project relationship with issues + + Ensure that your Jira project blueprint has a relationship with the Jira issue blueprint + + ```yaml showLineNumbers + relations: + project: .fields.project.key + ``` + ::: + +2. Navigate to the service blueprint and add this mirror property: + +
+ Highest priority issues count as CFR (click to expand) + +```json showLineNumbers + "changeFailureRate": { + "title": "Change Failure Rate Jira", + "path": "jiraProject.highestPriorityIssueCount" + } +``` +
+ + + + + +The “hotfix” approach interprets any pull request labeled as “hotfix” as representing a deployment that had to be rapidly fixed after release, effectively signaling a failed change. + +You can add this mapping to the service configuration to track hotfix deployments: + +
+Hotfix Deployment Mapping (click to expand) + +```yaml showLineNumbers +- kind: pull-request + selector: + query: '.labels | map(select(.name == "hotfix")) | length > 0' + port: + entity: + mappings: + identifier: .head.repo.name + '-' + (.id|tostring) + title: .head.repo.name + " Hotfix Deployment" + blueprint: '"deployment"' + properties: + createdAt: .merged_at + deploymentStatus: '"Failed"' + +``` +
+ + + + + + + + + + + + + +
### Monorepo Tracking @@ -1053,7 +1235,6 @@ To learn more about using search relations, see [our documentation on Mapping Re Using Search Queries](/build-your-software-catalog/customize-integrations/configure-mapping/#mapping-relations-using-search-queries). ::: - ## Metrics We will now aggregate the **DORA metrics** at the **service level**, allowing us to track metrics such as **Deployment Frequency**, **Change Lead Time**, **Change Failure Rate (CFR)**, and **Mean Time to Recovery (MTTR)** for each service. @@ -1278,8 +1459,6 @@ At this point, you can already visit each service to view the aggregated DORA me click [here](/build-your-software-catalog/customize-integrations/configure-data-model/setup-blueprint/properties/aggregation-property) for more details on aggregation properties. ::: - - ## Visualization By leveraging Port's Dashboards, you can create custom dashboards to track the metrics and monitor your team's performance over time. From 600e485d34b413eb2cfd9efb8c1f87a337e20aa4 Mon Sep 17 00:00:00 2001 From: kodjomiles Date: Mon, 16 Dec 2024 23:11:59 +0000 Subject: [PATCH 3/6] Update DORA metrics guide with improved tracking approaches Refined the DORA metrics setup guide by reworking the structure and adding alternative tracking methods, including Jira-based deployment tracking, lead time tracking, CFR using high-priority bugs, and hotfix deployments. Improved section clarity and alignment to recommended practices for tracking incidents and metrics. --- docs/guides/all/setup-dora-metrics.md | 531 +++++++++++++------------- 1 file changed, 258 insertions(+), 273 deletions(-) diff --git a/docs/guides/all/setup-dora-metrics.md b/docs/guides/all/setup-dora-metrics.md index e83959919..fbf031689 100644 --- a/docs/guides/all/setup-dora-metrics.md +++ b/docs/guides/all/setup-dora-metrics.md @@ -11,10 +11,10 @@ import PortTooltip from "/src/components/tooltip/tooltip.jsx"; This guide is designed to help you -implement and track [DevOps Research and Assessment (DORA) metrics](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance) within your organization in Port. +implement and track [DevOps Research and Assessment (DORA) metrics](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance) within your organization in Port. DORA Metrics are a set of key performance indicators -that measure the effectiveness and efficiency of your software development and delivery process. +that measure the effectiveness and efficiency of your software development and delivery process. By tracking these metrics, you can identify areas for improvement and ensure that your team is delivering high-quality software efficiently. This guide will cover the four key metrics: **deployment frequency**, **lead time**, **change failure rate**, and **mean time to recovery**. @@ -26,7 +26,6 @@ This guide will cover the four key metrics: **deployment frequency**, **lead tim - Optional for advanced strategies: If you're using workflows or pipelines, ensure they are configured for deployment tracking by following the relevant setup guides, such as CI/CD integrations or your platform-specific tools. - Optional for alternate tracking strategies: Install the [Port Jira integration](/build-your-software-catalog/sync-data-to-catalog/project-management/jira) to track deployments, lead time and change failure rate using Jira. - ## Tracking deployments In this section, we will cover how to track your team's deployments. Deployments refer to releasing new or updated code into various environments, such as **Production**, **Staging**, or **Testing**. Tracking deployments helps you understand how efficiently your team ships features and monitors release stability. @@ -109,15 +108,49 @@ To track the necessary data for these metrics, we will create a **Deployment Blu ``` -:::note Missing Lead Time +:::info Missing Lead Time + +We calculate the lead time using the **lead time** property in the git provider. If you do not have the **lead time** configured, you can follow the integration guide for your Git provider to add this property: - **GitHub**: [GitHub integration guide](/build-your-software-catalog/sync-data-to-catalog/git/github/examples/resource-mapping-examples#map-repositories-and-pull-requests) - **GitLab**: [GitLab integration guide](/build-your-software-catalog/sync-data-to-catalog/git/gitlab/examples#mapping-projects-readmemd-and-merge-requests) -- **Azure Repos**: [Azure Repos integration guide](/build-your-software-catalog/sync-data-to-catalog/git/azure-devops/examples#mapping-repositories-file-contents-repository-policies-and-pull-requests) +- **Azure Repos**: [Azure Repos integration guide](/build-your-software-catalog/sync-data-to-catalog/git/azure-devops/examples#mapping-repositories-file-contents-repository-policies-and-pull-requests) ::: +### Tracking lead time with Jira tickets + +The **recommended** approach in tracking the lead time is using measuring the time that a pr was opened to the time it was merged. +However, you can also track lead time by measuring the time that a Jira ticket moves from **In Progress** to **Done**. + +Add this mapping config to the Jira [data source](https://app.getport.io/settings/data-sources?filter=Jira): + +
+ Jira tickets config (click to expand) + +```yaml showLineNumbers +- kind: issue + selector: + query: 'true' + jql: 'status in ("In Progress", "Done")' + port: + entity: + mappings: + identifier: .key + title: .fields.summary + blueprint: '"jiraIssue"' + properties: + leadTimeHours: > + (.fields.created as $created | .fields.resolutiondate as $resolved | + if $resolved == null then null else + ((($resolved | sub("\\.[0-9]+\\+\\d{4}"; "") | strptime("%Y-%m-%dT%H:%M:%S") | mktime) - + ($created | sub("\\.[0-9]+\\+\\d{4}"; "") | strptime("%Y-%m-%dT%H:%M:%S") | mktime)) / 3600) end) +``` +
+ +This approach focuses on the entire lifecycle of a task rather than just code merges. + :::tip Adding JSON Schema Using Port's UI 1. **Go to the [Builder](https://app.getport.io/settings/data-model)** in your Port portal. 2. **Click on "+ Blueprint"** to create a new blueprint. @@ -127,19 +160,11 @@ If you do not have the **lead time** configured, you can follow the integration By following these steps, you can paste and manage the JSON schema required to track DORA metrics in Port. ::: - ### Tracking strategies Below are the main ways you can track deployments directly within Port: - - - - + - + - Below is a demonstration of how deployment tracking can be implemented using the PR merge strategy. +Below is a demonstration of how deployment tracking can be implemented using the PR merge strategy. - **Example**: +**Example**: - - When a PR is merged, a **deployment entity** is created in Port to represent the deployment that took place. - - The **lead time** for that PR is calculated and added to the deployment as part of the blueprint. +- When a PR is merged, a **deployment entity** is created in Port to represent the deployment that took place. +- The **lead time** for that PR is calculated and added to the deployment as part of the blueprint. - Here’s how you can implement this: +Here’s how you can implement this: - 1. **Add Pull Request blueprint, sample can be found [here](/build-your-software-catalog/sync-data-to-catalog/git/github/examples/resource-mapping-examples/#map-repositories-and-pull-requests)**. +1. **Add Pull Request blueprint, sample can be found [here](/build-your-software-catalog/sync-data-to-catalog/git/github/examples/resource-mapping-examples/#map-repositories-and-pull-requests)**. - 2. **Add the configuration below** to the [data sources page](https://app.getport.io/settings/data-sources) in your Port portal, and select your GitHub integration: +2. **Add the configuration below** to the [data sources page](https://app.getport.io/settings/data-sources) in your Port portal, and select your GitHub integration:
Deployment config (click to expand) @@ -203,18 +228,18 @@ The lead time for these merges is calculated as the difference between when the - Below is a demonstration of how deployment tracking can be implemented using the MR merge strategy: +Below is a demonstration of how deployment tracking can be implemented using the MR merge strategy: - **Example**: +**Example**: - - When a MR is merged, a **deployment entity** is created in Port to represent the deployment that took place. - - The **lead time** for that MR is calculated and added to the deployment as part of the blueprint. +- When a MR is merged, a **deployment entity** is created in Port to represent the deployment that took place. +- The **lead time** for that MR is calculated and added to the deployment as part of the blueprint. - Here’s how you can implement this: +Here’s how you can implement this: - 1. **Add Merge Request blueprint, sample can be found [here](/build-your-software-catalog/sync-data-to-catalog/git/gitlab/examples#mapping-projects-readmemd-and-merge-requests)**. +1. **Add Merge Request blueprint, sample can be found [here](/build-your-software-catalog/sync-data-to-catalog/git/gitlab/examples#mapping-projects-readmemd-and-merge-requests)**. - 2. **Add the configuration below** to the [data sources page](https://app.getport.io/settings/data-sources) in your Port portal, and select your GitLab integration: +2. **Add the configuration below** to the [data sources page](https://app.getport.io/settings/data-sources) in your Port portal, and select your GitLab integration:
Deployment config (click to expand) @@ -312,28 +337,28 @@ You can modify these values based on your requirements. Track deployments by monitoring workflow runs in your pipeline. -This setup captures all workflow runs from the main branch and maps them to deployment entities. +This setup captures all workflow runs from the main branch and maps them to deployment entities. The deployment status is set dynamically based on whether the workflow run concluded successfully or failed. Below is a demonstration of how deployment tracking can be implemented using workflow/job. - + - Below is how the deployment tracking can be done in **GitHub** using workflow runs: +Below is how the deployment tracking can be done in **GitHub** using workflow runs: - **Example**: +**Example**: - - When a GitHub Actions workflow runs on the main branch, a **deployment entity** is created in Port to represent the deployment. - - The **deployment status** is dynamically set based on the workflow run's conclusion. +- When a GitHub Actions workflow runs on the main branch, a **deployment entity** is created in Port to represent the deployment. +- The **deployment status** is dynamically set based on the workflow run's conclusion. - Here’s how you can implement this: +Here’s how you can implement this: - 1. **Add the configuration below** to the [data sources page](https://app.getport.io/settings/data-sources) in your Port portal, and select your **GitHub** integration: +1. **Add the configuration below** to the [data sources page](https://app.getport.io/settings/data-sources) in your Port portal, and select your **GitHub** integration:
Deployment via Workflow Runs (click to expand) @@ -364,16 +389,16 @@ Below is a demonstration of how deployment tracking can be implemented using wor - Below is how the deployment tracking can be done in **GitLab** using pipeline jobs: +Below is how the deployment tracking can be done in **GitLab** using pipeline jobs: - **Example**: +**Example**: - - When a GitLab CI/CD pipeline runs on the main branch, a **deployment entity** is created in Port. - - The **deployment status** is dynamically set based on the pipeline's status. +- When a GitLab CI/CD pipeline runs on the main branch, a **deployment entity** is created in Port. +- The **deployment status** is dynamically set based on the pipeline's status. - Here’s how you can implement this: +Here’s how you can implement this: - 1. **Add the configuration below** to the [data sources page](https://app.getport.io/settings/data-sources) in your Port portal, and select your **GitLab** integration: +1. **Add the configuration below** to the [data sources page](https://app.getport.io/settings/data-sources) in your Port portal, and select your **GitLab** integration:
Deployment via Pipeline Runs (click to expand) @@ -418,6 +443,7 @@ By following a similar pattern for other tools, you will be able to capture depl + CI/CD pipelines, such as those run by Jenkins, provide a robust way to track deployments. Jenkins, in particular, allows you to create and update entities in Port dynamically using Port's API as part of the pipeline execution. @@ -426,23 +452,23 @@ CI/CD pipelines, such as those run by Jenkins, provide a robust way to track dep Port supports tracking deployments from various CI/CD tools by monitoring pipelines and reporting the deployment status to Port. Here are examples for some commonly used CI/CD tools: - - + + - - Jenkins provides a robust way to track deployments by dynamically reporting build statuses to Port using Port's API. [find more here](/build-your-software-catalog/custom-integration/api/ci-cd/jenkins-deployment/) - Add this example to your Jenkins pipeline to report deployments to Port: +Jenkins provides a robust way to track deployments by dynamically reporting build statuses to Port using Port's API. [find more here](/build-your-software-catalog/custom-integration/api/ci-cd/jenkins-deployment/) + +Add this example to your Jenkins pipeline to report deployments to Port:
Jenkins Pipeline Example (click to expand) - + ```groovy showLineNumbers pipeline { agent any @@ -508,19 +534,19 @@ Port supports tracking deployments from various CI/CD tools by monitoring pipeli } } ``` - +
- Octopus can be used to track deployments by reporting to Port using custom API calls after deployments are triggered. [Find more here](/build-your-software-catalog/sync-data-to-catalog/cicd/octopus-deploy) - - Add this example to your Octopus script to report deployments to Port: +Octopus can be used to track deployments by reporting to Port using custom API calls after deployments are triggered. [Find more here](/build-your-software-catalog/sync-data-to-catalog/cicd/octopus-deploy) + +Add this example to your Octopus script to report deployments to Port:
Octopus Deploy Example (click to expand) - + ```yaml # Octopus script example for deployment tracking in Port steps: @@ -551,19 +577,19 @@ Port supports tracking deployments from various CI/CD tools by monitoring pipeli }' \ https://api.getport.io/v1/blueprints/deployment/entities?upsert=true&merge=true ``` - +
- Track deployments in CircleCI by reporting pipeline runs to Port using a configuration similar to Jenkins. [Find more here](/build-your-software-catalog/custom-integration/api/ci-cd/circleci-workflow/) - - Add this example to your CircleCI pipeline to report deployments to Port: +Track deployments in CircleCI by reporting pipeline runs to Port using a configuration similar to Jenkins. [Find more here](/build-your-software-catalog/custom-integration/api/ci-cd/circleci-workflow/) + +Add this example to your CircleCI pipeline to report deployments to Port:
CircleCI Pipeline Example (click to expand) - + ```yaml version: 2.1 @@ -645,20 +671,20 @@ Port supports tracking deployments from various CI/CD tools by monitoring pipeli }' \ https://api.getport.io/v1/blueprints/deployment/entities?upsert=true&merge=true ``` - +
- + - Track deployments in Codefresh by integrating Port's API with your pipeline configurations. [find more here](/build-your-software-catalog/custom-integration/api/ci-cd/codefresh-workflow-template/) +Track deployments in Codefresh by integrating Port's API with your pipeline configurations. [find more here](/build-your-software-catalog/custom-integration/api/ci-cd/codefresh-workflow-template/) - Add this example to your Codefresh pipeline to report deployments to Port: +Add this example to your Codefresh pipeline to report deployments to Port:
Codefresh Pipeline Example (click to expand) - + ```yaml version: '1.0' steps: @@ -691,18 +717,18 @@ Port supports tracking deployments from various CI/CD tools by monitoring pipeli } }' ``` - +
- Track GitLab pipeline deployments using a similar approach to report data to Port. [find more here](/build-your-software-catalog/custom-integration/api/ci-cd/gitlab-pipelines/) - - Add this example to your GitLab pipeline to report deployments to Port: +Track GitLab pipeline deployments using a similar approach to report data to Port. [find more here](/build-your-software-catalog/custom-integration/api/ci-cd/gitlab-pipelines/) + +Add this example to your GitLab pipeline to report deployments to Port:
GitLab Pipeline Example (click to expand) - + ```yaml stages: - deploy @@ -800,6 +826,7 @@ Unlike GitHub, **GitLab** and **Azure DevOps** do not support traditional releas + If your tool or workflow is not natively supported, you can create custom integrations by directly interacting with Port’s API. This method allows you to track deployments from any system that can make HTTP API calls. @@ -848,189 +875,13 @@ and remember to replace `$YOUR_PORT_API_TOKEN` with your actual API token from p By using this approach, you can ensure any deployment system is integrated into Port, giving you full flexibility across all your deployment workflows. - - - - - - - - -Although PR merge is the recommended way of tracking deployments, besides the CI methods described above, -you can also track and measure DORA metrics using other approaches that may align better with your team's workflow. - - - - - -Instead of relying solely on PR merges or pipeline runs, you can treat a Jira ticket moving to the "Done" status as an indicator that code has effectively been deployed. - -
-Track deployments based on Jira ticket status (click to expand) - -```yaml showLineNumbers -- kind: issue - selector: - # Use a JQL or a query that filters for issues with status = Done - jql: 'status = Done' - port: - entity: - mappings: - identifier: .key - title: .fields.summary - blueprint: '"deployment"' - properties: - # Using the issue's creation time as the deployment time - createdAt: .fields.created - # Marking all completed issues as successful deployments - deploymentStatus: '"Success"' -``` - -
- - - - - -Calculate lead time based on how long it takes a Jira ticket to move from “In Progress” to “Done.” -This approach focuses on a more business-oriented measure of lead time, tying it directly to ticket lifecycle rather than just code merges. - -
-Calculate lead time on Jira ticket status (click to expand) - -```yaml showLineNumbers -
- -```yaml showLineNumbers -- kind: issue - selector: - jql: 'status in ("In Progress", "Done")' - port: - entity: - mappings: - identifier: .key - title: .fields.summary - blueprint: '"jiraIssue"' - properties: - # Calculate lead time in hours from created to resolutionDate - leadTimeHours: > - (.fields.created as $created | .fields.resolutiondate as $resolved | - if $resolved == null then null else - ((($resolved | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) - - ($created | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime)) / 3600) end) -``` - -
- - - - - -Count the number of urgent Jira bugs that appear after deployments to gauge the change failure rate. -This method aligns failure rate with customer-impacting issues tracked in your ticketing system. - -To implement this follow the steps below: - -1. Add an aggregation property to the Jira `project` blueprint to count high-priority issues on a project: - -
-Aggregation property for high-priority issues (click to expand) - -```json showLineNumbers -"highestPriorityIssueCount": { -"title": "Highest Priority Issue Count", -"target": "jiraIssue", -"calculationSpec": { - "calculationBy": "entities", - "func": "count" -}, -"query": { - "combinator": "and", - "rules": [ - { - "property": "priority", - "operator": "=", - "value": "Highest" - } - ] -} -} - -``` -
- - :::tip Project relationship with issues - - Ensure that your Jira project blueprint has a relationship with the Jira issue blueprint - - ```yaml showLineNumbers - relations: - project: .fields.project.key - ``` - ::: - -2. Navigate to the service blueprint and add this mirror property: - -
- Highest priority issues count as CFR (click to expand) - -```json showLineNumbers - "changeFailureRate": { - "title": "Change Failure Rate Jira", - "path": "jiraProject.highestPriorityIssueCount" - } -``` -
- - - - - -The “hotfix” approach interprets any pull request labeled as “hotfix” as representing a deployment that had to be rapidly fixed after release, effectively signaling a failed change. - -You can add this mapping to the service configuration to track hotfix deployments: - -
-Hotfix Deployment Mapping (click to expand) - -```yaml showLineNumbers -- kind: pull-request - selector: - query: '.labels | map(select(.name == "hotfix")) | length > 0' - port: - entity: - mappings: - identifier: .head.repo.name + '-' + (.id|tostring) - title: .head.repo.name + " Hotfix Deployment" - blueprint: '"deployment"' - properties: - createdAt: .merged_at - deploymentStatus: '"Failed"' - -``` -
- - - - - - - - -
- -### Monorepo Tracking +### Monorepo tracking By using **custom integrations**, you can effectively track services or components within a monorepo. Here’s how you can do this: @@ -1069,10 +920,49 @@ By this method, individual services within a monorepo are mapped to Port bluepri Custom integrations provide flexibility in mapping and tracking each service or microservice within a monorepo. With Port’s API, you can track deployments and updates for each component separately, giving you granular control over monitoring and managing services in a monorepo. ::: +## Alternative tracking strategies + +Although PR merge is the recommended way of tracking deployments, besides the CI methods described above, +you can also track and measure DORA metrics using other approaches that may align better with your team's workflow. + + +### Deployment via Jira Tickets + +Instead of relying solely on PR merges or pipeline runs, +you can treat a Jira ticket moving to the "Done" status as indicating that code has been deployed. +To implement this, add this to your Jira data source configuration: + +:::tip Install jira integration +You need to make sure that you have a Jira integration set up in Port with the Issue blueprint to track Jira tickets. +If you don't have this set up, follow the [Jira integration guide](/build-your-software-catalog/sync-data-to-catalog/issue-tracking/jira) to configure it. +::: + +Add this mapping config to the Jira [data source](https://app.getport.io/settings/data-sources?filter=Jira) + +```yaml showLineNumbers +- kind: issue + selector: + query: 'true' + jql: 'status = Done' + port: + entity: + mappings: + identifier: .key + title: .fields.summary + blueprint: '"deployment"' + properties: + createdAt: .fields.created + deploymentStatus: '"Success"' +``` + +This approach shifts the deployment definition from a code-centric event (PR merge) to a work-management event (Jira issue completion). + + ## Tracking incidents -Incidents are essential for tracking key DORA metrics, including **Change Failure Rate (CFR)** and **Mean Time to Recovery (MTTR)**. Effective incident tracking reveals insights into how frequently deployments fail and how quickly teams can resolve issues. This section outlines how to: +Incidents are essential for tracking key DORA metrics, including **Change Failure Rate (CFR)** and **Mean Time to Recovery (MTTR)**. +Effective incident tracking reveals insights into how frequently deployments fail and how quickly teams can resolve issues. This section outlines how to: - Use incidents to calculate **CFR** and **MTTR**. - Link incidents to services to track the impact of failures. @@ -1080,14 +970,14 @@ Incidents are essential for tracking key DORA metrics, including **Change Failur ### Data model setup -Ensure that your **PagerDuty incident blueprint** is properly configured to map incidents to the correct services(gitHub). +Ensure that your **PagerDuty incident blueprint** is properly configured to map incidents to the correct services(gitHub). This includes defining the appropriate properties and relations for incidents.Follow this [PagerDuty Incident Blueprint Setup Link](/build-your-software-catalog/sync-data-to-catalog/incident-management/pagerduty#incident-blueprint-setup) to implement. - Add the following properties to capture incident resolution time and recovery time:
Additional properties for PagerDuty Incident Blueprint (click to expand) - + ```json showLineNumbers "resolvedAt": { "title": "Incident Resolution Time", @@ -1108,7 +998,7 @@ This includes defining the appropriate properties and relations for incidents.Fo
Incident mapping config for resolvedAt and recoveryTime (click to expand) - + ```yaml showLineNumbers resolvedAt: .resolved_at recoveryTime: >- @@ -1117,7 +1007,7 @@ This includes defining the appropriate properties and relations for incidents.Fo ( ($resolvedAt | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) - ($createdAt | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) ) / 60 end) # Time in minutes and divide by 3600 if you want it calculated in hours ``` - +
### Syncing incidents with PagerDuty and other tools @@ -1232,15 +1122,107 @@ Update the mapping config to pagerduty incident [data source](https://app.getpor we use the **search relation** entity to map the `pagerdutyIncident` blueprint to the correct service based on the service's `$title` and the pagerduty incident `service.summary`. We have assumed that the **service name/title** exists in the `service.summary` property of the incident, but you can modify this query to map based on other properties that better match your setup To learn more about using search relations, see [our documentation on Mapping Relations -Using Search Queries](/build-your-software-catalog/customize-integrations/configure-mapping/#mapping-relations-using-search-queries). +Using Search Queries](/build-your-software-catalog/customize-integrations/configure-mapping/#mapping-relations-using-search-queries). ::: +Note that the above implementation of tracking incidents using an incident manager tool like Pagerduty and others is the **recommended** approach. + + +### Tracking CFR with Jira bugs (alternative) + +Count the number of urgent Jira bugs that appear after deployments to gauge the change failure rate. +This method aligns failure rate with customer-impacting issues tracked in your ticketing system. + +To implement this follow the steps below: + +1. Add an aggregation property to the Jira `project` blueprint to count high-priority issues on a project: + +
+Aggregation property for high-priority issues (click to expand) + +```json showLineNumbers +"highestPriorityIssueCount": { +"title": "Highest Priority Issue Count", +"target": "jiraIssue", +"calculationSpec": { + "calculationBy": "entities", + "func": "count" +}, +"query": { + "combinator": "and", + "rules": [ + { + "property": "priority", + "operator": "=", + "value": "Highest" + } + ] +} +} + +``` +
+ +:::tip Project relationship with issues + +Ensure that your Jira project blueprint has a relationship with the Jira issue blueprint + + ```yaml showLineNumbers + relations: + project: .fields.project.key + ``` +::: + +2. Navigate to the service blueprint and add this mirror property: + +
+ Highest priority issues count as CFR (click to expand) + +```json showLineNumbers + "changeFailureRate": { + "title": "Change Failure Rate Jira", + "path": "jiraProject.highestPriorityIssueCount" + } +``` +
+ +### Tracking hotfix deployments (alternative) +An Alternate way of tracking incidents is to use the hotfix approach. +The “hotfix” approach interprets any pull request labeled as “hotfix” +as representing a deployment that had to be rapidly fixed after release, effectively signaling a failed change. + +Add this mapping to the service configuration to track hotfix deployments: + +
+Hotfix Deployment Mapping (click to expand) + +```yaml showLineNumbers +- kind: pull-request + selector: + query: '.labels | map(select(.name == "hotfix")) | length > 0' + port: + entity: + mappings: + identifier: .head.repo.name + '-' + (.id|tostring) + title: .head.repo.name + " Hotfix Deployment" + blueprint: '"deployment"' + properties: + createdAt: .merged_at + deploymentStatus: '"Failed"' + +``` +
+ ## Metrics We will now aggregate the **DORA metrics** at the **service level**, allowing us to track metrics such as **Deployment Frequency**, **Change Lead Time**, **Change Failure Rate (CFR)**, and **Mean Time to Recovery (MTTR)** for each service. :::note Metrics Aggregation -If you want to track these metrics at higher levels, such as **team** or **domain**, makes sure the appropriate **team** or **domain** blueprints exist, and that they have relationships defined with the **service** blueprint. Then, you can apply the aggregation properties for these higher hierarchies, similar to how we are doing for the **service** blueprint below. +If you want to track these metrics at higher levels, such as **team** or **domain**, +makes sure the appropriate **team** or **domain** blueprints exist, +and that they have relationships defined with the **service** blueprint. +Then, you can apply the aggregation properties for these higher hierarchies, +similar to how we are doing for the **service** blueprint below. ::: ### Aggregation @@ -1255,6 +1237,7 @@ Before proceeding, follow these steps to add the aggregation and calculation pro 3. Click the `{...}` button in the top right corner, and choose **Edit JSON**. 4. Insert the respective **aggregation** or **calculation properties** under the `aggregationProperties` or `calculationProperties` section in the Service blueprint's JSON schema. 5. Save your changes to apply the new aggregation configuration. + ::: @@ -1397,7 +1380,7 @@ Add the following to the aggregated property in service: Add this calculation property to calculate the cfr from the aggregated properties:
CFR calculation property (click to expand) - + ```json showLineNumbers "changeFailureRate": { "title": "Change Failure Rate", @@ -1459,6 +1442,8 @@ At this point, you can already visit each service to view the aggregated DORA me click [here](/build-your-software-catalog/customize-integrations/configure-data-model/setup-blueprint/properties/aggregation-property) for more details on aggregation properties. ::: + + ## Visualization By leveraging Port's Dashboards, you can create custom dashboards to track the metrics and monitor your team's performance over time. @@ -1468,7 +1453,7 @@ By leveraging Port's Dashboards, you can create custom dashboards to track the m 3. Select **New dashboard**. 4. Name the dashboard (e.g., DORA Metrics), choose an icon if desired, and click `Create`. -This will create a new empty dashboard. Let's get ready-to-add widgets +This will create a new empty dashboard. Let's get ready-to-add widgets ### Adding widgets @@ -1481,7 +1466,7 @@ This will create a new empty dashboard. Let's get ready-to-add widgets 4. Select an `Entity` and choose `Monthly Deployment Frequency` as the **Property**. - + 5. Click `Save`.
@@ -1495,7 +1480,7 @@ This will create a new empty dashboard. Let's get ready-to-add widgets 4. Select an `Entity` and choose `Mean Time to Recovery` as the **Property**. - + 5. Click `Save`.
@@ -1511,7 +1496,7 @@ This will create a new empty dashboard. Let's get ready-to-add widgets 5. Add custom Unit for **Unit of Measurement** (Hours). - + 6. Click `Save`.
From b0348a9c9ea08e473f1025fd422ccddffc35f3df Mon Sep 17 00:00:00 2001 From: kodjomiles Date: Mon, 16 Dec 2024 23:40:19 +0000 Subject: [PATCH 4/6] Update link path in Jira integration guide reference The link to the Jira integration guide was incorrect. Updated the path from "issue-tracking" to "project-management" to ensure accurate navigation for users. --- docs/guides/all/setup-dora-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/all/setup-dora-metrics.md b/docs/guides/all/setup-dora-metrics.md index fbf031689..036633d5a 100644 --- a/docs/guides/all/setup-dora-metrics.md +++ b/docs/guides/all/setup-dora-metrics.md @@ -934,7 +934,7 @@ To implement this, add this to your Jira data source configuration: :::tip Install jira integration You need to make sure that you have a Jira integration set up in Port with the Issue blueprint to track Jira tickets. -If you don't have this set up, follow the [Jira integration guide](/build-your-software-catalog/sync-data-to-catalog/issue-tracking/jira) to configure it. +If you don't have this set up, follow the [Jira integration guide](/build-your-software-catalog/sync-data-to-catalog/project-management/jira) to configure it. ::: Add this mapping config to the Jira [data source](https://app.getport.io/settings/data-sources?filter=Jira) From bf513e15749257aacd5452bf282791d068cb150a Mon Sep 17 00:00:00 2001 From: kodjomiles Date: Wed, 18 Dec 2024 14:31:57 +0000 Subject: [PATCH 5/6] Revised DORA metrics guide to clarify alternate lead time tracking approaches using Jira. Enhanced incident tracking section with alternative methods, tabs for clarity, and improved formatting. These changes aim to improve usability and comprehension without altering functionality. --- docs/guides/all/setup-dora-metrics.md | 69 ++++++++++++++++++++------- 1 file changed, 53 insertions(+), 16 deletions(-) diff --git a/docs/guides/all/setup-dora-metrics.md b/docs/guides/all/setup-dora-metrics.md index 036633d5a..b0b24a28e 100644 --- a/docs/guides/all/setup-dora-metrics.md +++ b/docs/guides/all/setup-dora-metrics.md @@ -122,7 +122,8 @@ If you do not have the **lead time** configured, you can follow the integration ### Tracking lead time with Jira tickets The **recommended** approach in tracking the lead time is using measuring the time that a pr was opened to the time it was merged. -However, you can also track lead time by measuring the time that a Jira ticket moves from **In Progress** to **Done**. +An **alternate** approach is to track lead time using Jira tickets from their **creation** until **resolution**. +This approach focuses on the entire lifecycle of a task rather than just code merges. Add this mapping config to the Jira [data source](https://app.getport.io/settings/data-sources?filter=Jira): @@ -149,7 +150,11 @@ Add this mapping config to the Jira [data source](https://app.getport.io/setting ```
-This approach focuses on the entire lifecycle of a task rather than just code merges. +If you want to measure lead time from the moment a ticket moves **In Progress** rather than from creation, set up an automation to update a custom field in Port when the status changes to In Progress, +and then use a calculation property in Port to measure the time between this custom field’s timestamp and the resolved date. +Read more about setting automations [here](/actions-and-automations/define-automations/). + + :::tip Adding JSON Schema Using Port's UI 1. **Go to the [Builder](https://app.getport.io/settings/data-model)** in your Port portal. @@ -920,13 +925,7 @@ By this method, individual services within a monorepo are mapped to Port bluepri Custom integrations provide flexibility in mapping and tracking each service or microservice within a monorepo. With Port’s API, you can track deployments and updates for each component separately, giving you granular control over monitoring and managing services in a monorepo. ::: -## Alternative tracking strategies - -Although PR merge is the recommended way of tracking deployments, besides the CI methods described above, -you can also track and measure DORA metrics using other approaches that may align better with your team's workflow. - - -### Deployment via Jira Tickets +### Tracking deployment via Jira Tickets (Alternative) Instead of relying solely on PR merges or pipeline runs, you can treat a Jira ticket moving to the "Done" status as indicating that code has been deployed. @@ -968,7 +967,20 @@ Effective incident tracking reveals insights into how frequently deployments fai - Link incidents to services to track the impact of failures. - Aggregate metrics across incidents for better monitoring. -### Data model setup +Track incident by following the ways below: + + + + + +We recommend using a dedicated incident management tool, like PagerDuty, to streamline this process. + +Below is an example of how to do this + +

Data model setup

Ensure that your **PagerDuty incident blueprint** is properly configured to map incidents to the correct services(gitHub). This includes defining the appropriate properties and relations for incidents.Follow this [PagerDuty Incident Blueprint Setup Link](/build-your-software-catalog/sync-data-to-catalog/incident-management/pagerduty#incident-blueprint-setup) to implement. @@ -1010,7 +1022,7 @@ This includes defining the appropriate properties and relations for incidents.Fo
-### Syncing incidents with PagerDuty and other tools +

Syncing incidents with PagerDuty and other tools

To sync incidents from **PagerDuty**, follow the steps in the [PagerDuty guide](/build-your-software-catalog/sync-data-to-catalog/incident-management/pagerduty). The guide provides detailed steps for setting up integrations to track incidents related to deployments. @@ -1020,7 +1032,8 @@ For other incident management tools, follow these respective guides: - [ServiceNow](/build-your-software-catalog/sync-data-to-catalog/incident-management/servicenow) - [Statuspage](/build-your-software-catalog/sync-data-to-catalog/incident-management/statuspage) -### Relating Incident to services +

Relating Incident to services

+ Add this relationship to the **Incident blueprint** to link incidents to GitHub or Gitlab or Azure DevOps repository (service): + + + + +There are different ways to track incidents depending on your tools and needs. Below are a couple of alternate +strategies that can complement the recommended methods: + + + -### Tracking CFR with Jira bugs (alternative) +

Tracking CFR with Jira bugs

Count the number of urgent Jira bugs that appear after deployments to gauge the change failure rate. This method aligns failure rate with customer-impacting issues tracked in your ticketing system. @@ -1184,10 +1211,13 @@ Ensure that your Jira project blueprint has a relationship with the Jira issue b "path": "jiraProject.highestPriorityIssueCount" } ``` -
+ + + -### Tracking hotfix deployments (alternative) -An Alternate way of tracking incidents is to use the hotfix approach. +

Tracking hotfix deployments

+ +Another way of tracking incidents is to use the hotfix approach. The “hotfix” approach interprets any pull request labeled as “hotfix” as representing a deployment that had to be rapidly fixed after release, effectively signaling a failed change. @@ -1212,6 +1242,13 @@ Add this mapping to the service configuration to track hotfix deployments: ``` + + + + + + + ## Metrics From 903f159c1b8cee291f47d554f637b57c06fc4e48 Mon Sep 17 00:00:00 2001 From: kodjomiles Date: Wed, 18 Dec 2024 14:32:42 +0000 Subject: [PATCH 6/6] renamed header. --- docs/guides/all/setup-dora-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/all/setup-dora-metrics.md b/docs/guides/all/setup-dora-metrics.md index b0b24a28e..c9e8d3f96 100644 --- a/docs/guides/all/setup-dora-metrics.md +++ b/docs/guides/all/setup-dora-metrics.md @@ -925,7 +925,7 @@ By this method, individual services within a monorepo are mapped to Port bluepri Custom integrations provide flexibility in mapping and tracking each service or microservice within a monorepo. With Port’s API, you can track deployments and updates for each component separately, giving you granular control over monitoring and managing services in a monorepo. ::: -### Tracking deployment via Jira Tickets (Alternative) +### Tracking deployment via Jira tickets (Alternative) Instead of relying solely on PR merges or pipeline runs, you can treat a Jira ticket moving to the "Done" status as indicating that code has been deployed.