Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Audit some integration tests, make minor changes, and add plaintext test list #2089

Merged
merged 96 commits into from
Oct 10, 2024

Conversation

p-offtermatt
Copy link
Contributor

@p-offtermatt p-offtermatt commented Jul 23, 2024

Description

Adds a script that generates a nice list of scenarios that are tested via integration tests.

Added missing docstrings and improved existing ones for several tests, along with a few minor fixes.


Extra notes:

This function interchain-security/testutil/keeper/unit_test_helpers.go
should use rapid instead. With rand, we only get one attempt at this per run, and it's impossible to trace back the original parameters if it ever fails.

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • Included the correct type prefix in the PR title
  • Targeted the correct branch (see PR Targeting)
  • Provided a link to the relevant issue or specification
  • Reviewed "Files changed" and left comments if necessary
  • Confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • Confirmed the correct type prefix in the PR title
  • Confirmed all author checklist items have been addressed
  • Confirmed that this PR does not change production code

Summary by CodeRabbit

  • New Features

    • Introduced a GitHub Actions workflow to automate testing documentation updates.
    • Added a new target in the Makefile for testing documentation.
    • Implemented a Go program to extract and document test function descriptions automatically.
    • Enhanced test documentation for various integration tests in the blockchain system.
  • Bug Fixes

    • Improved test coverage for misbehavior detection and reward distribution mechanisms.
  • Documentation

    • Expanded TESTING.md to provide clearer guidelines for documenting integration tests, including schema requirements.
    • Updated documentation for key tests to improve clarity and ensure better understanding.
  • Chores

    • Cleaned up unused imports and streamlined type references in test files.

@github-actions github-actions bot added C:Testing Assigned automatically by the PR labeler C:Build Assigned automatically by the PR labeler labels Jul 23, 2024
@github-actions github-actions bot added the C:CI Assigned automatically by the PR labeler label Jul 23, 2024
@github-actions github-actions bot added the C:x/provider Assigned automatically by the PR labeler label Jul 23, 2024
@p-offtermatt p-offtermatt marked this pull request as ready for review July 23, 2024 14:03
@p-offtermatt p-offtermatt requested a review from a team as a code owner July 23, 2024 14:03
Copy link
Contributor

coderabbitai bot commented Jul 23, 2024

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough

Walkthrough

The recent updates introduce a comprehensive GitHub Actions workflow to maintain testing documentation, enhance Makefile functionality for documentation testing, and significantly improve integration test documentation. Multiple new tests have been added to validate key functionalities, particularly related to blockchain operations, including rewards distribution, validator behavior, and misbehavior detection. These enhancements ensure better clarity in documentation and robust testing coverage, ultimately improving the reliability and maintainability of the codebase.

Changes

File(s) Change Summary
.github/workflows/testing-docs.yml Introduced a GitHub Actions workflow for maintaining up-to-date testing documentation, triggered by various events and capable of modifying documentation based on changes in integration tests.
Makefile Added a new target testing-docs to facilitate testing of documentation by executing a Go script.
TESTING.md Enhanced documentation regarding integration tests, including a new section on test scenario documentation and instructions for updating the test documentation.
scripts/test_doc/extract_docstrings.go Introduced a Go program to automate the extraction of documentation strings from test functions, generating a markdown file with detailed descriptions.
scripts/test_doc/test_documentation.md Created a new documentation resource outlining various integration tests for a blockchain system, presented in a structured table format.
tests/integration/changeover.go Added a new test, TestRecycleTransferChannel, to validate the reuse of transfer channels during chain transitions.
tests/integration/channel_init.go Expanded documentation for TestInitTimeout to clarify its purpose and behavior during handshake processes.
tests/integration/democracy.go Introduced three new tests to enhance coverage of the democracy module, validating rewards distribution and governance proposals.
tests/integration/distribution.go Updated comments for existing tests and added a new test, TestAllocateTokensToConsumerValidators, to check token distribution logic.
tests/integration/key_assignment.go Enhanced the TestKeyAssignment function with a new helper function for validating key assignments, improving test coverage and robustness.
tests/integration/misbehaviour.go Added two new tests, TestGetByzantineValidators and TestCheckMisbehaviour, to validate misbehavior detection logic.
tests/integration/provider_gov_hooks.go Introduced a new test function, TestGetConsumerAdditionLegacyPropFromProp, to assess proposal handling for consumer additions.
tests/integration/slashing.go Added TestSlashPacketAcknowledgement to validate the handling of slash packet acknowledgements.
tests/integration/throttle.go Introduced TestDoubleSignDoesNotAffectThrottling to ensure throttling remains stable under double sign slashing scenarios.
testutil/keeper/unit_test_helpers.go Modified the GetNewSlashPacketData function to change the source for computing the Infraction field.
x/ccv/provider/keeper/relay_test.go Removed an unused import and changed type references for clarity in the test function.

Sequence Diagram(s)

sequenceDiagram
    participant Developer
    participant GitHubActions as GA
    participant Makefile as MF
    participant Docs as D
    participant Tests as T

    Developer->>GA: Trigger events (PR, push)
    GA->>MF: Execute testing-docs workflow
    MF->>T: Run tests on integration changes
    GA->>D: Update documentation if needed
    D-->>Developer: Notify documentation status
Loading

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 16

Outside diff range, codebase verification and nitpick comments (13)
.github/workflows/testing-docs.yml (1)

42-52: Step for checking for changes is appropriate but consider improving error messaging.

The step correctly compares the generated documentation with the committed version and exits with an error if they differ. Consider adding more descriptive error messages for better clarity.

-  echo "Documentation for integration tests is out of date. Please run `make testing-docs`!"
+  echo "Error: Documentation for integration tests is out of date. Please run `make testing-docs` to update it."
scripts/test_doc/extract_docstrings.go (1)

64-79: Docstring extraction logic is appropriate but consider using os.ReadFile.

The extractDocstrings function reads the Go source file, creates the AST, and traverses it to extract docstrings. This logic is appropriate. However, consider using os.ReadFile instead of ioutil.ReadFile as ioutil is deprecated.

-  src, err := ioutil.ReadFile(filePath)
+  src, err := os.ReadFile(filePath)
tests/integration/slashing.go (8)

Line range hint 1-1:
Ensure consistent import aliasing.

The import alias for evidencetypes is redundant since it matches the package name.

-	evidencetypes "cosmossdk.io/x/evidence/types"
+	"github.com/cosmos/cosmos-sdk/x/evidence/types"

Line range hint 53-53:
Check for nil error instead of using NoError.

Using NoError is fine, but explicitly checking for nil error is more readable in some cases.

-	s.Require().Nil(err)
+	s.Require().NoError(err)

Line range hint 155-155:
Check for nil error instead of using NoError.

Using NoError is fine, but explicitly checking for nil error is more readable in some cases.

-	s.Require().Nil(err)
+	s.Require().NoError(err)

258-262: Improve the function description.

The long description placeholder should be replaced with a detailed explanation of the test's purpose.

-// @Long Description@
+// This test ensures that a slash packet sent from a consumer chain is properly acknowledged by the provider chain.

Line range hint 320-320:
Check for nil error instead of using NoError.

Using NoError is fine, but explicitly checking for nil error is more readable in some cases.

-	suite.Require().Nil(err)
+	suite.Require().NoError(err)

Line range hint 388-388:
Check for nil error instead of using NoError.

Using NoError is fine, but explicitly checking for nil error is more readable in some cases.

-	suite.Require().Nil(err)
+	suite.Require().NoError(err)

Line range hint 571-571:
Check for nil error instead of using NoError.

Using NoError is fine, but explicitly checking for nil error is more readable in some cases.

-	suite.Require().Nil(err)
+	suite.Require().NoError(err)

Line range hint 725-725:
Check for nil error instead of using NoError.

Using NoError is fine, but explicitly checking for nil error is more readable in some cases.

-	suite.Require().Nil(err)
+	suite.Require().NoError(err)
tests/integration/distribution.go (3)

23-31: Improve the function description.

The long description placeholder should be replaced with a detailed explanation of the test's purpose.

-// @Long Description@
+// This test verifies the correct distribution of rewards from the consumer chain to the provider chain.
+// It sets up the necessary channels, sends tokens to the fee pool, and checks their distribution.
+// The test ensures that tokens are correctly allocated to the consumer and provider chains,
+// and verifies the handling of rewards that are not whitelisted.

194-200: Improve the function description.

The long description placeholder should be replaced with a detailed explanation of the test's purpose.

-// @Long Description@
+// This test verifies the retry mechanism for failed reward transmissions.
+// It sets up the necessary channels, fills the fee pool, and intentionally corrupts the transmission channel.
+// The test then advances the block height to trigger a retry and confirms the successful transmission of rewards.

850-860: Improve the function description.

The long description placeholder should be replaced with a detailed explanation of the test's purpose.

-// @Long Description@
+// This test verifies the allocation of tokens to consumer validators under various scenarios.
+// It sets up the provider chain, initializes consumer validators, and calls the AllocateTokensToConsumerValidators function.
+// The test checks the expected token allocation to validators and covers different scenarios such as empty tokens,
+// empty consumer validator set, single validator allocation, and multiple validator allocation.

tests/integration/provider_gov_hooks.go Outdated Show resolved Hide resolved
TESTING.md Show resolved Hide resolved
scripts/test_doc/test_documentation.md Outdated Show resolved Hide resolved
scripts/test_doc/test_documentation.md Outdated Show resolved Hide resolved
scripts/test_doc/test_documentation.md Outdated Show resolved Hide resolved
scripts/test_doc/test_documentation.md Outdated Show resolved Hide resolved
scripts/test_doc/test_documentation.md Outdated Show resolved Hide resolved
scripts/test_doc/test_documentation.md Outdated Show resolved Hide resolved
scripts/test_doc/test_documentation.md Outdated Show resolved Hide resolved
scripts/test_doc/test_documentation.md Outdated Show resolved Hide resolved
@p-offtermatt p-offtermatt changed the title test: Audit all integration tests, make minor changes, and add plaintext test list test: Audit some integration tests, make minor changes, and add plaintext test list Jul 23, 2024
@p-offtermatt
Copy link
Contributor Author

@coderabbitai pause

@p-offtermatt
Copy link
Contributor Author

@coderabbitai resolve

.github/workflows/testing-docs.yml Show resolved Hide resolved
.github/workflows/testing-docs.yml Outdated Show resolved Hide resolved
.github/workflows/testing-docs.yml Show resolved Hide resolved
.github/workflows/testing-docs.yml Outdated Show resolved Hide resolved
.github/workflows/testing-docs.yml Show resolved Hide resolved
scripts/test_doc/extract_docstrings.go Show resolved Hide resolved
scripts/test_doc/extract_docstrings.go Show resolved Hide resolved
tests/integration/key_assignment.go Outdated Show resolved Hide resolved
tests/integration/key_assignment.go Show resolved Hide resolved
Copy link
Contributor

@mpoke mpoke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/integration/slashing.go Show resolved Hide resolved
tests/integration/slashing.go Outdated Show resolved Hide resolved
tests/integration/slashing.go Show resolved Hide resolved
@mpoke mpoke added this pull request to the merge queue Oct 10, 2024
Merged via the queue into main with commit acbc4c7 Oct 10, 2024
16 of 18 checks passed
@mpoke mpoke deleted the ph/test-table branch October 10, 2024 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C:Build Assigned automatically by the PR labeler C:CI Assigned automatically by the PR labeler C:Testing Assigned automatically by the PR labeler C:x/provider Assigned automatically by the PR labeler C:x/types Assigned automatically by the PR labeler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants