Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve XCM Debugging by Capturing Logs in Unit Tests #7594

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

raymondkfcheung
Copy link

@raymondkfcheung raymondkfcheung commented Feb 17, 2025

Description

This PR introduces a lightweight log-capturing mechanism for XCM unit tests, making it easier to troubleshoot failures when needed. It partially addresses #6119 and #6125 by providing an optional way to verify log outputs in tests.

TODOs

  • Introduces a structured way to capture logs for XCM debugging.
  • Ensures logging remains unobtrusive in passing tests but is available when debugging.

Review Notes:

  • This PR does not enforce logging in all tests but provides a reusable mechanism for capturing logs when necessary.
  • The implementation is minimal and does not add complexity to existing test setups.
  • Developers can still rely on standard logging, sp_tracing::init_for_tests(), when manual debugging is sufficient.

@raymondkfcheung raymondkfcheung requested a review from a team as a code owner February 17, 2025 14:58
@raymondkfcheung raymondkfcheung self-assigned this Feb 17, 2025
@raymondkfcheung raymondkfcheung marked this pull request as draft February 17, 2025 14:58
@paritytech-workflow-stopper
Copy link

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/13373075940
Failed job name: fmt

@raymondkfcheung raymondkfcheung changed the title Assert on captured logs on XCM Simulator Improve XCM Debugging by Capturing Logs in Unit Tests Feb 17, 2025
@raymondkfcheung raymondkfcheung added the T6-XCM This PR/Issue is related to XCM. label Feb 17, 2025
@raymondkfcheung
Copy link
Author

Currently, there is no straightforward way to verify if log messages are correctly output during unit tests.

I initially explored using tracing-test, but it only captures logs at the global scope (see source). This means that when testing inside Relay::execute_with(|| { ... }), the logs are not captured in global_buf(), making it unsuitable for verifying log output in such cases.

For sp_tracing::try_init_simple() and sp_tracing::init_for_tests(), they are designed to enable log output but do not provide a way to capture logs for assertions in tests.

Given this, I've implemented a custom log capture mechanism within sp_tracing to support verifying logs in unit tests, particularly for XCM-related tests. Let me know if you have any thoughts on this approach!

@acatangiu
Copy link
Contributor

Given this, I've implemented a custom log capture mechanism within sp_tracing to support verifying logs in unit tests, particularly for XCM-related tests. Let me know if you have any thoughts on this approach!

if this is simple, then sure, it's nice to have, otherwise I don't think we really need to verify this in tests - we have many devs acting as canaries 😆

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T6-XCM This PR/Issue is related to XCM.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants