[new_relic sink] unexpected backslash inserted before dot for attribute keys

[alfie029]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

I used Vector.dev to forward my logs to NewRelic with newrelic sink, it worked well with the basic features, but not good if there are dots in attribute keys, for example, NewRelic log content have some specified attributes which keys are like entity.guid, entity.name, entity.type, trace.id, span.id and also my customized attribute key with same pattern (for example, given dotnet.nrerelic.logs.osversion ), all those key was changed with a backslash before dot, for example

dotnet\.newrelic\.logs\.osversion:"Unix 5.15.49.0"
entity\.name:"dotnet-newrelic-log-forward"
entity\.type:"SERVICE"

those messages would break the linking from logs to APM transactions

Here is a snapshot of newrelic logs query page

and the console output like below (use console sink)

{"@assembly":"dotnet-newrelic-log-forward, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null","@l":"Information","@mt":"Content root path: {contentRoot}","@t":"2022-11-16T08:43:05.6213200Z","SourceContext":"Microsoft.Hosting.Lifetime","contentRoot":"/app/","dotnet.newrelic.logs.osversion":"Unix 5.15.49.0","entity.name":"dotnet-newrelic-log-forward","entity.type":"SERVICE","forward":"vector","hostname":"66eda19e2961","level":"Information","message":"Content root path: \"/app/\""}

Configuration

#                                    __   __  __
#                                    \ \ / / / /
#                                     \ V / / /
#                                      \_/  \/
#
#                                    V E C T O R
#                                   Configuration
#
# ------------------------------------------------------------------------------
# Website: https://vector.dev
# Docs: https://vector.dev/docs
# Chat: https://chat.vector.dev
# ------------------------------------------------------------------------------

# Change this to use a non-default directory for Vector data storage:
# data_dir = "/var/lib/vector"

[sources.dotnet]
  # REQUIRED - General
  type = "exec"
  command = ["dotnet", "/app/dotnet-newrelic-log-forward.dll"]
  mode = "streaming"

# See the Vector Remap Language reference for more info: https://vrl.dev
[transforms.parse_logs]
  # REQUIRED
  type = "remap"
  inputs = ["dotnet"]
  source = """
. = parse_json!(.message)
.forward = "vector" 
"""

# Print parsed logs to stdout
[sinks.newrelic]
  # REQUIRED
  type = "new_relic"
  inputs = [ "parse_logs" ]
  account_id = "${NEW_RELIC_ACCOUNT_ID}"
  license_key = "${NEW_RELIC_LICENSE_KEY}" # example, no default
  api = "logs"
  # OPTIONAL
  region = "us"
  
[sinks.console]
  type = "console"
  inputs = [ "parse_logs" ]
  target = "stdout"
  
  [sinks.console.encoding]
    codec = "json"

# Vector's GraphQL API (disabled by default)
# Uncomment to try it out with the `vector top` command or
# in your browser at http://localhost:8686
#[api]
#enabled = true
#address = "127.0.0.1:8686"

Version

vector 0.25.0 (x86_64-unknown-linux-gnu b573b51 2022-11-01)

Debug Output

No response

Example Data

{"@assembly":"dotnet-newrelic-log-forward, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null","@l":"Information","@mt":"Content root path: {contentRoot}","@t":"2022-11-16T08:43:05.6213200Z","SourceContext":"Microsoft.Hosting.Lifetime","contentRoot":"/app/","dotnet.newrelic.logs.osversion":"Unix 5.15.49.0","entity.name":"dotnet-newrelic-log-forward","entity.type":"SERVICE","forward":"vector","hostname":"66eda19e2961","level":"Information","message":"Content root path: \"/app/\""}

Additional Context

No response

References

No response

[mayanksingh2298]

Hey is there any plan to sort this out?

@alfie029 I'm in a similar boat. As a workaround right now, I've been using http_sink insted of new relic sink with json encoding. Something like this:

new_relic_sink:
    type: http
    inputs:
      - ctrlb-newrelic-input
    uri: https://log-api.newrelic.com/log/v1
    buffer:
      max_events: 30000
      type: memory
    request:
      headers:
        X-License-Key : xxx
        Content-Type : application/json
    encoding:
      codec: json

Do you have a better solution?

[bruceg]

This behavior actually changed in #21201, which went into versions 0.40.2 and 0.41.0. The current behavior no longer adds the \ escaping but rather ends up quoting dotted paths, which might not be the correct behavior neither.

Could you test with one of the above recent releases to see if the behavior is resolved for you?

[bruceg]

This issue should be resolved in #21305. If you are still encountering problems, please open a new issue with the details.