newrelic & sidekiq 7.2.1 agent injection not working?

[Insood]

Description

I'm trying to upgrade our Rails 6.1 application to the latest newrelic_rpm (9.7.1) & sidekiq gem versions so that we can upgrade to Rails 7.1.

We have the sidekiq web UI mounted in our application and I'm running into a minor issue where sidekiq does not allow for injecting inline scripts since 7.2 (sidekiq/sidekiq#6074). I'm seeing errors in console along the lines of Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'self' https: http:". Either the 'unsafe-inline' keyword, a hash ('sha256-eg5bTdCwlsbd2Ex1VrxAGTrpwcYjEfeImwhVCzs1uJw='), or a nonce ('nonce-...') is required to enable inline execution.

This script appears to be a newrelic script:

        <script type="text/javascript">
            window.NREUM || (NREUM = {});
            NREUM.info = {
                "beacon": "bam.nr-data.net",
                "errorBeacon": "bam.nr-data.net",

As far as I can tell all this is affecting is that the browser will not any data from the sidekiq web UI to newrelic.

Expected Behavior

No console errors or some way to configure new relic to not inject the browser agent at specific paths.

Troubleshooting or NR Diag results

This issue does not happen with the previous minor release of sidekiq (7.1) leaving me to believe this is due to the above referenced sidekiq security update

Steps to Reproduce

Please be as specific as possible.
TIP: Link a sample application that demonstrates the issue.

Your Environment

Ruby 3.1.2
Rails 6.1.7.6
newrelic_rpm 9.7.1
sidekiq 7.2.1
sidekiq-ent 7.2.0
sidekiq-pro 7.2.0

common: &default_settings
  license_key: abc123

uat:
  <<: *default_settings
  app_name: uat.ourapp.com
  log_level: info
  instrumentation:
    net_http: chain
  distributed_tracing:
    enabled: true
  monitor_mode: true
  application_logging:
    forwarding:
      enabled: true
  attributes.include: job.sidekiq.args.*

Additional context

N/A

For Maintainers Only or Hero Triaging this bug

N/A

[workato-integration]

https://new-relic.atlassian.net/browse/NR-221594

[kaylareopelle]

Thank you for opening this bug report, @Insood! We're taking a look at the issue and will get back to you no later than Wednesday with next steps.

[kaylareopelle]

Hi @Insood! I'm able to reproduce this bug. Thanks for the detailed description! 🎉

We have a few ideas about how to resolve this issue. Since browser agent injection is evaluated on almost every Rack request, we want to make sure we have a performant solution.

While we're working on the fix, if you'd like to stop the error, you could disable browser monitoring by updating your configuration to include:

  browser_monitoring.auto_instrument: false

Unfortunately, this'll stop browser monitoring for your entire application, not just Sidekiq's Web UI.

[fallwith]

Hi @Insood,

We the maintainers of this repo had a chat with the team that maintains the browser agent. For at least the near future, the agent will need to continue to use an inline script tag and therefore the web app generating the HTML will need to include the unsafe-inline exception. Today I learned that we have this requirement documented under the Content Security Policy considerations section of our documentation. Given the dynamic nature of the delivery of the browser agent, using a nonce or hosting content beforehand with a CDN will unfortunately not work.

I've learned that the issue of unsafe-inline has come up internally at New Relic from time to time, and we continue to explore our options to strike the best balance between secure and convenient while leaving a minimal footprint on modifying an observed application. For now we're sticking with the inline tag, and for some internal bookkeeping we've removed the 'bug' label from this issue.

Going forward, this GitHub issue will be used to explore having the agent offer more granular options for determining to either include or exclude the script tag on a contextual basis. For example, perhaps we'll use an allowlist or denylist of routes or perhaps we'll detect a Sidekiq context to exclude the tag, etc. If you have any preferences or suggestions for this type of granularity, do please let us know.

In the meantime before any dev work is completed and released for this one, one option you have is to use an environment variable based override of the YAML based agent configuration.

For example, if you have a web app on host A and Sidekiq on host B, you could set NEW_RELIC_BROWSER_MONITORING_AUTO_INSTRUMENT=false on host B to disable the insertion of the script tag on only that host.

[fallwith]

Hi @Insood,

With #2544, a community member enhanced the agent to support the uses of nonces for the <script> tag that was causing problems here with Sidekiq's CSP settings.

If you upgrade the New Relic Ruby agent to v9.10.2 or higher, it will attempt to include a nonce from ActionDispatch when injecting the <script> tag for the browser agent. If you're using Sidekiq with Rails, it's possible that it will address this issue and permit you to use the browser agent for Sidekiq web UI traffic.