Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Datadog support for span-related monitoring functions #419

Closed
19 of 21 tasks
timmc-edx opened this issue May 30, 2024 · 3 comments
Closed
19 of 21 tasks

Add Datadog support for span-related monitoring functions #419

timmc-edx opened this issue May 30, 2024 · 3 comments
Assignees

Comments

@timmc-edx
Copy link
Contributor

timmc-edx commented May 30, 2024

[Consider breaking out each method as its own ticket when this is to be picked up.]

Acceptance criteria:

  • Complete all of the following for each method:
    • function_trace: feat: Create manual spans for monitoring backends. #435
      • Move implementation into NewRelicBackend.
      • Implement in DatadogBackend.
      • Update function to call all of the telemetry backends.
      • edx_django_utils/monitoring/README.rst updated with latest compatibility info.
      • Inform owners of calling code, especially for backward-incompatible changes in Datadog.
    • set_monitoring_transaction_name
      • Move implementation into NewRelicBackend.
      • Implement in DatadogBackend.
      • Update function to call all of the telemetry backends.
      • edx_django_utils/monitoring/README.rst updated with latest compatibility info.
      • Inform owners of calling code, especially for backward-incompatible changes in Datadog (see notes).
    • ignore_transaction
      • Move implementation into NewRelicBackend.
      • Implement in DatadogBackend.
      • Update function to call all of the telemetry backends.
      • edx_django_utils/monitoring/README.rst updated with latest compatibility info.
      • Inform owners of calling code, especially for backward-incompatible changes in Datadog.
    • Handle get_current_transaction appropriately (see notes).
    • Handle background_task appropriately (see notes).

Implementation notes:

  • Optionally, choose new names that are less specific to NR, and deprecate the old names (leaving stubs in their place that emit a warning and call the new function).

Other notes:

  • Some repos currently call these directly on newrelic.agent rather than using edx-django-utils's indirection. [APM] Remove direct references to newrelic edx/edx-arch-experiments#621 covers resolving this.
  • It appears that get_current_transaction is currently only used internally, despite being exposed, so it can probably be skipped for now.
  • Relatedly, background_task appears to be completely unused. (Should this be retired?)
  • 2U did not end up using OpenTelemetry, so we can't commit to adding support to OpenTelemetryBackend at the moment. These new methods can be implemented as no-ops in that backend.
  • The method set_monitoring_transaction_name is used for XBlocks here, and the resource name will probably change, which may affect monitoring or other XBlock related observability.
@robrap
Copy link
Contributor

robrap commented Aug 7, 2024

Note that function_trace has since been implemented.

@robrap
Copy link
Contributor

robrap commented Aug 21, 2024

Additional Notes:

  • What remains is NR transaction-related calls: set_monitoring_transaction_name and ignore_transaction (found in transactions.py)
  • The closest thing to a NR transaction in Datadog (and OpenTelemtry) is the Span.
  • In Datadog, we probably want set_monitoring_transaction_name to update the resource name only (not operation name). I think the more general name goes to both the resource and operation name.

UPDATE: We need to communicate these changes.

@robrap
Copy link
Contributor

robrap commented Aug 21, 2024

I updated AC and moved this to P3, since these methods are in use. It would be great to confirm and retire what is not in use(see notes).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants