feat(config): support libdatadog's library_config for config through file

[BaptisteFoy]

Description of changes

• Bumps libdatadog to v15.0.0 & handles the required changes to crashtracking
• Starts using the library_config component of libdatadog that allows reading configuration from a file with a higher precedence than environment variables

Supported config entries

As of libdatadog v15.0.0 (source):

• DD_TRACE_DEBUG
• DD_SERVICE
• DD_ENV
• DD_VERSION
• DD_PROFILING_ENABLED

Testing strategy

Added a unit test.

To test this PR, you can:

1. Create a Linux VM
2. Start this tracer version & ensure everything works well (non-regression)
3. Create a file at the following path: /etc/datadog-agent/managed/datadog-apm-libraries/stable/libraries_config.yaml
4. Write the following content in the file:

rules:
  - selectors:
      - origin: language
        matches:
          - python
        operator: equals
    configuration:
      DD_TRACE_DEBUG: true

5. Start this tracer version
6. Verify that debug logs are properly enabled

Additional concerns

This change does increase the size of the serverless layer by 380KB (14204KB vs 13824 as the limit) even after optimization. I had to go from lto=true to lto=fast, as lto=true somehow prevents release builds on older machines. That accounts for 200KB, while the 200 other KB are mostly coming from the libdatadog bump.

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

src/native/library_config.rs                                            @DataDog/apm-core-python
tests/internal/test_native.py                                           @DataDog/apm-core-python
ddtrace/__init__.py                                                     @DataDog/apm-core-python
ddtrace/internal/native/__init__.py                                     @DataDog/apm-core-python
ddtrace/internal/native/_native.pyi                                     @DataDog/apm-core-python
src/native/Cargo.lock                                                   @DataDog/apm-core-python
src/native/Cargo.toml                                                   @DataDog/apm-core-python
src/native/lib.rs                                                       @DataDog/apm-core-python

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-01-24 16:23:27

Comparing candidate commit 1a40d17 in PR branch baptiste.foy/FA/library-config-libdatadog with baseline commit 9e87349 in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 392 metrics, 2 unstable metrics.

scenario:iast_aspects-ospathbasename_aspect

• 🟥 execution_time [+293.691ns; +345.716ns] or [+8.822%; +10.385%]

scenario:iast_aspects-ospathsplit_aspect

• 🟥 execution_time [+381.109ns; +441.813ns] or [+9.967%; +11.555%]

[taegyunkim]

👋 I'd like to suggest not adding new modules under /profiling as the functionality is not profiling. You should be able to expose the libdatadog API through src/core, or add a new rust/python module under src/

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: baptiste.foy/FA/library-config-libdatadog
Commit report: 1575326
Test service: dd-trace-py

✅ 0 Failed, 208 Passed, 1390 Skipped, 5m 3.12s Total duration (35m 0.17s time saved)

[taegyunkim]

Adding @brettlangdon to the reviewers early so you could get feedback sooner than later

[brettlangdon]

can we break up this PR?

I think there is the libdatadog version bump, core package renaming, and then the config file changes which can all get broken up.

[mabdinur]

Overall looks great! High level comments:

• Setting environment variables directly is good enough for this POC now but we will need to revisit this design decision to better track the origin of configurations.
• We need to identify all scenarios where the value set by stable configurations can be overriden by the ddtrace library.
• We need to document a clear precedence for configurations and ensure all libraries follow it (rc -> stable_config -> in_code -> envar -> reading from yaml file)
• We need to document which configurations are supported by 15.0. Including this in the PR description is fine.
• We should split up this PR (@brettlangdon's comment)

[mabdinur]

In the python tracer the DD_SERVICE environment variable can be overridden by integrations and other ddtrace products (ex: serverless).

We will need to audit all the configurations supported by libdatadog and ensure the value set via APM Stable Configurations take precedence.

[BaptisteFoy]

Agreed. I also think that we need to have a discussion on the order of precedence. For instance in the agent, config values set by the runtime (ie inside the agent code) take precedence over Fleet Automation. One could argue that we should replicate that behaviour here; but imo this is a product decision.

[BaptisteFoy]

@brettlangdon @mabdinur Could you please review #12063 & #12065 as new prerequisites to this PR, as suggested?

[mabdinur]

LGTM. Once the prerequisite PRs are merged we can ship this change