false WARN messages after extension upgrade

[Vladikamira]

Hi, as I mentioned here after an upgrade the datadog lambda extension to version v64 we got weird messages popped up in the logs

2024-09-11 15:09:38 UTC | DD_EXTENSION | WARN | config key flare_stripped_keys is unknown
2024-09-11 15:09:38 UTC | DD_EXTENSION | WARN | failed to get configuration value for key "flare_stripped_keys": unable to cast <nil> of type <nil> to []string
2024-09-11 15:09:38 UTC | DD_EXTENSION | WARN | config key scrubber.additional_keys is unknown
2024-09-11 15:09:38 UTC | DD_EXTENSION | WARN | failed to get configuration value for key "scrubber.additional_keys": unable to cast <nil> of type <nil> to []string

we don't have any of these configurations configured, therefore I assume it's a bug

here is our config:

apm_config:
  enabled: true

site: datadoghq.eu

use_dogstatsd: false

cloud_provider_metadata:
  - "aws"

# https://docs.datadoghq.com/opentelemetry/otlp_ingest_in_the_agent/?tab=host
otlp_config:
  receiver:
    protocols:
      grpc:
        endpoint: 127.0.0.1:50052
  metrics:
    enabled: true
    resource_attributes_as_tags: true
    tag_cardinality: low
  traces:
    enabled: true
    span_name_remappings:
      go.opentelemetry.io_contrib_instrumentation_google.golang.org_grpc_otelgrpc.server: grpc.server
      go.opentelemetry.io_contrib_instrumentation_google.golang.org_grpc_otelgrpc.client: grpc.client
      go.opentelemetry.io_contrib_instrumentation_net_http_otelhttp.client: http.client
      go.opentelemetry.io_contrib_instrumentation_net_http_otelhttp.server: http.server
      observability_lambda.server: aws.lambda
      aws.lambda.internal: aws.lambda

logs_enabled: true
logs_config:
  processing_rules:
    - type: exclude_at_match
      name: exclude_start_and_end_logs
      pattern: (START|END|REPORT)\s

I did tried to add empty list to the config

scrubber:
  additional_keys: []

but it didn't solve the problem

[duncanista]

Hey @Vladikamira,

Is this on v64? If so, could you try upgrading to v65?
If the problem persist, let me know!

[Vladikamira]

v65 has the same issue

[duncanista]

Is it the only one? If so, does removing

I did tried to add empty list to the config

scrubber:
  additional_keys: []

This from your config file solves it?

[Vladikamira]

no, nothing is solving these messages. Here is what I have on v64/v65

2024-09-16 07:34:40 UTC | DD_EXTENSION | WARN | config key fleet_policies_dir is unknown
2024-09-16 07:34:40 UTC | DD_EXTENSION | WARN | failed to get configuration value for key "scrubber.additional_keys": unable to cast <nil> of type <nil> to []string
2024-09-16 07:34:40 UTC | DD_EXTENSION | WARN | failed to get configuration value for key "flare_stripped_keys": unable to cast <nil> of type <nil> to []string
2024-09-16 07:34:40 UTC | DD_EXTENSION | WARN | failed to get configuration value for key "scrubber.additional_keys": unable to cast <nil> of type <nil> to []string
2024-09-16 07:34:40 UTC | DD_EXTENSION | WARN | config key scrubber.additional_keys is unknown
2024-09-16 07:34:40 UTC | DD_EXTENSION | WARN | failed to get configuration value for key "flare_stripped_keys": unable to cast <nil> of type <nil> to []string
2024-09-16 07:34:40 UTC | DD_EXTENSION | WARN | config key flare_stripped_keys is unknown

[duncanista]

Thanks @Vladikamira will look more into it

[eugenepavliy]

Hi. Is there any update on this? Having same issue on arm v65.

[shenma6]

any update? V65 is having the same issue

[duncanista]

I'll assign someone on Monday to look over this, sorry haven't been able to reproduce this.

[nhulston]

Hello @Vladikamira

Thank you for opening this. This has been fixed in DataDog/datadog-agent#30622. We plan on doing a Lambda extension release this week.

[Vladikamira]

Awesome! Thanks for the update @nhulston , I will wait for the release and check

[nhulston]

Hey @Vladikamira, apologies for coming back to this. That PR I linked doesn't actually fix this issue. Sorry for the confusion. However, we've identified the real issue and are working on a fix. The change should be out soon!

[nhulston]

This has been fixed in DataDog/datadog-agent#31087 and will be resolved in the next release!

[duncanista]

Hey everyone,

Sorry for the delay.
The latest release v66 should include this fix.

Feel free to re-open if issue persist!