Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable NR everywhere asap to reduce costs #701

Closed
1 of 3 tasks
robrap opened this issue Jun 27, 2024 · 3 comments
Closed
1 of 3 tasks

Disable NR everywhere asap to reduce costs #701

robrap opened this issue Jun 27, 2024 · 3 comments

Comments

@robrap
Copy link
Contributor

robrap commented Jun 27, 2024

We should disable NR everywhere to reduce costs. Targeting the biggest problems first is probably best, but we also don't want the slow leak of money either.

  • Disabling NR APM causes trace concatenation in Datadog #692 (hack should take care of edxapp - in-progress)
    • Note: Disabling edxapp will make the rest more clear.
    • We were able to stop edxapp from reporting data to NR, so the costs side of this issue is complete. (There are other problems that that issue still covers.)
  • See Slack thread for some of the biggest culprits for frontend.
  • We can get updated numbers from Data Management (may need Admin) in NR.

Notes:

  • As of June 26, the following spreadsheet has us spending about $350/day on NR storage.
  • The Data Mangement page provides NRQL for each section. More NRQL could be provided, or we could make our own dashboard. For example:
SELECT bytecountestimate()/10e8 FROM `Metric` WHERE instrumentation.provider != 'kentik' AND instrumentation.provider != 'pixie' FACET CASES (WHERE newrelic.source = 'agent' AS 'Metric Time Slices', WHERE newrelic.source != 'agent' OR newrelic.source IS NULL AS 'Dimensional Metrics') SINCE 1 hour ago

Or

SELECT bytecountestimate()/10e8 FROM `Metric` WHERE newrelic.source = 'agent' AND instrumentation.provider != 'kentik' AND instrumentation.provider != 'pixie' FACET appName LIMIT 15 SINCE 1 hour ago
@robrap
Copy link
Contributor Author

robrap commented Jun 27, 2024

@spencertiberi: This could get your support for some of the follow-up with other teams to help disable things.

@timmc-edx
Copy link
Member

Costs reduced enough that we can push this out further. Remaining work is lower priority.

@dianakhuang
Copy link
Member

We are no longer concerned about costs. Closing this for now.

@dianakhuang dianakhuang closed this as not planned Won't fix, can't repro, duplicate, stale Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

3 participants