You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When the unit tests run, errors and warnings may appear in either the New Relic agent log and/or the Rails log when Rails is in scope.
Sometimes these errors are expected, because behavior that would produce a warning or an error was artificially induced under test.
Sometimes these errors are not expected, and unless a human reviews the log(s), the errors and warnings will go unnoticed.
For improved developer visibility of unexpected errors and for increased scrutiny performed by test automation, it would be ideal if errors and warnings that are found in the logs will be surfaced to the human viewer and cause a test failure when under automation.
Here's a proposal for a linear sequence of passes to take:
After a test completes, quote any WARN and ERROR lines from newrelic_agent.log to STDOUT. These should be hidden unless VERBOSE_TEST_OUTPUT is enabled for now.
After a test completes, also quote any WARN and ERROR lines from the relevant Rails log to STDOUT. These should be hidden unless VERBOSE_TEST_OUTPUT is enabled for now.
Create an "expect this action to log with this severity" type helper that will monitor our own internal logger class's behavior AND also mark any generated log file entries to be ignored and not output to STDOUT.
Once 1-3 are satisfied, consider the presence of a WARN or ERROR message to be cause for a test failure. Fail the test and output the log message(s). Do this regardless of VERBOSE_TEST_OUTPUT.
The text was updated successfully, but these errors were encountered:
When the unit tests run, errors and warnings may appear in either the New Relic agent log and/or the Rails log when Rails is in scope.
Sometimes these errors are expected, because behavior that would produce a warning or an error was artificially induced under test.
Sometimes these errors are not expected, and unless a human reviews the log(s), the errors and warnings will go unnoticed.
For improved developer visibility of unexpected errors and for increased scrutiny performed by test automation, it would be ideal if errors and warnings that are found in the logs will be surfaced to the human viewer and cause a test failure when under automation.
Here's a proposal for a linear sequence of passes to take:
VERBOSE_TEST_OUTPUT
is enabled for now.VERBOSE_TEST_OUTPUT
is enabled for now.VERBOSE_TEST_OUTPUT
.The text was updated successfully, but these errors were encountered: