Skip to content

feat: Backend for SDK metrics #5623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

feat: Backend for SDK metrics #5623

wants to merge 8 commits into from

Conversation

khvn26
Copy link
Member

@khvn26 khvn26 commented Jun 19, 2025

Thanks for submitting a PR! Please check the boxes below:

  • I have added information to docs/ if required so people know about the feature!
  • I have filled in the "Changes" section below?
  • I have filled in the "How did you test this code" section below?
  • I have used a Conventional Commit title for this Pull Request

Changes

Closes #5581.

Contributes to #2479.

This PR adds support for optional header tracking in the backend. It adds extensible, fully typed mechanism to support other optional labels for analytics in the future. The usage data and feature evaluation data routes are updated to return labeled data buckets. Additionally, numerous typing and code structure improvements are introduced to the analytics-related code base.

SDK API

All tracked SDK APIs now expect the following optional headers, translated to labels written to PostgreSQL or InfluxDB usage and evaluation data buckets:

  • Flagsmith-Application-Name header: client_application_name label
  • Flagsmith-Application-Version header: client_application_version label

Relevant caches and tasks are changed to properly compact the raw data, producing buckets with unique label sets.

Management API

The following API routes are updated to support labels:

  • /api/v1/organisations/{organisation_pk}/usage-data/
  • /api/v1/projects/{project_pk}/features/{id}/evaluation-data/

Previously, these routes were returning one bucket per day. Now, multiple buckets can be attributed to the same day, as long as they have different unique label sets. The label sets are returned under the label key for each bucket.

Both of them now accept new query parameters:

  • client_application_name
  • client_application_version

to allow bucket filtering by label.

Task processing

Due to incompatible signature change, the track_feature_evaluation is decommissioned in favour of the new track_feature_evaluations_by_environment task, which now supports labels included as part of feature evaluation data.

How did you test this code?

Unit tests have been added and updated to reflect new behaviour. Manually with local PostgreSQL storage. Additional tests will be conducted in staging against InfluxDB Cloud to measure the performance.

Copy link

vercel bot commented Jun 19, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

3 Skipped Deployments
Name Status Preview Comments Updated (UTC)
docs ⬜️ Ignored (Inspect) Visit Preview Jun 20, 2025 2:56pm
flagsmith-frontend-preview ⬜️ Ignored (Inspect) Visit Preview Jun 20, 2025 2:56pm
flagsmith-frontend-staging ⬜️ Ignored (Inspect) Visit Preview Jun 20, 2025 2:56pm

@github-actions github-actions bot added api Issue related to the REST API feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from a36fe70 to 2b706ee Compare June 19, 2025 10:02
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 2b706ee to 88498ed Compare June 19, 2025 10:26
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 88498ed to 421522b Compare June 19, 2025 11:01
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 421522b to 8664928 Compare June 19, 2025 11:16
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
Copy link

codecov bot commented Jun 19, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.80%. Comparing base (23aa20f) to head (eb0c370).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5623      +/-   ##
==========================================
+ Coverage   97.74%   97.80%   +0.06%     
==========================================
  Files        1255     1247       -8     
  Lines       44500    44221     -279     
==========================================
- Hits        43496    43251     -245     
+ Misses       1004      970      -34     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 8664928 to 2a67741 Compare June 19, 2025 14:41
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 force-pushed the feat/client-app-telemetry branch from 2a67741 to 02c72e2 Compare June 19, 2025 16:00
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 changed the title feat: Backend for client app telementry feat: Backend for client app telemetry Jun 19, 2025
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 marked this pull request as ready for review June 19, 2025 17:31
@khvn26 khvn26 requested a review from a team as a code owner June 19, 2025 17:31
@khvn26 khvn26 requested review from emyller and removed request for a team June 19, 2025 17:31
Copy link
Contributor

github-actions bot commented Jun 19, 2025

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-api-test:pr-5623 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-e2e:pr-5623 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api:pr-5623 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-frontend:pr-5623 Finished ✅ Results
ghcr.io/flagsmith/flagsmith:pr-5623 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-private-cloud:pr-5623 Finished ✅ Results

@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 19, 2025
@khvn26 khvn26 changed the title feat: Backend for client app telemetry feat: Backend for SDK metrics Jun 20, 2025
@github-actions github-actions bot added feature New feature or request and removed feature New feature or request labels Jun 20, 2025
Copy link
Contributor

@emyller emyller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines +212 to +217
if period in {constants.CURRENT_BILLING_PERIOD, constants.PREVIOUS_BILLING_PERIOD}:
if not (sub_cache and sub_cache.is_billing_terms_dates_set()):
raise NotFound("No billing periods found for this organisation.")

if TYPE_CHECKING:
assert sub_cache
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context aside, this feels a bit fragile. Can we make sure sub_cache is not None before this assert statement?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually are doing it by raising NotFound, but mypy is too dumb to understand that.

Copy link
Contributor

@emyller emyller Jun 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It raises only if period is found in that set, + if sub_cache.is_billing_terms_dates_set() evaluates to true, which from here feels like sub_cache could still be None at L216. Perhaps tweaking the logical expression could help mypy? This is a nit unless there's actual risk of an type/attribute error.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it raises if sub_cache is None. (sub_cache and sub_cache.is_billing_terms_dates_set()) evaluates to None, and not (sub_cache and sub_cache.is_billing_terms_dates_set()) evaluates to False.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I'm slow today — will fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Issue related to the REST API feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement API for tracking SDK metrics
2 participants