Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Improve logged error visibility #2263

Open
fallwith opened this issue Oct 16, 2023 · 1 comment
Open

CI: Improve logged error visibility #2263

fallwith opened this issue Oct 16, 2023 · 1 comment

Comments

@fallwith
Copy link
Contributor

When the unit tests run, errors and warnings may appear in either the New Relic agent log and/or the Rails log when Rails is in scope.

Sometimes these errors are expected, because behavior that would produce a warning or an error was artificially induced under test.

Sometimes these errors are not expected, and unless a human reviews the log(s), the errors and warnings will go unnoticed.

For improved developer visibility of unexpected errors and for increased scrutiny performed by test automation, it would be ideal if errors and warnings that are found in the logs will be surfaced to the human viewer and cause a test failure when under automation.

Here's a proposal for a linear sequence of passes to take:

  1. After a test completes, quote any WARN and ERROR lines from newrelic_agent.log to STDOUT. These should be hidden unless VERBOSE_TEST_OUTPUT is enabled for now.
  2. After a test completes, also quote any WARN and ERROR lines from the relevant Rails log to STDOUT. These should be hidden unless VERBOSE_TEST_OUTPUT is enabled for now.
  3. Create an "expect this action to log with this severity" type helper that will monitor our own internal logger class's behavior AND also mark any generated log file entries to be ignored and not output to STDOUT.
  4. Once 1-3 are satisfied, consider the presence of a WARN or ERROR message to be cause for a test failure. Fail the test and output the log message(s). Do this regardless of VERBOSE_TEST_OUTPUT.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants