Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tests): eof section order tests #592

Merged
merged 2 commits into from
Jun 12, 2024
Merged

feat(tests): eof section order tests #592

merged 2 commits into from
Jun 12, 2024

Conversation

winsvega
Copy link
Collaborator

@winsvega winsvega commented Jun 5, 2024

🗒️ Description

Construct eof container with skipped sections combinations

Example:

  • take code section, make it appear only in container header and header types, but skip printing bytes in the body
  • take code section, make it appear only in container body but skip printing bytes in the header
  • take code section but print it in a wrong order in header or body

🔗 Related Issues

#550

✅ Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
  • Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

Copy link
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments, thanks!

@marioevz
Copy link
Member

marioevz commented Jun 6, 2024

Another comment is that, since there are some repeated cases with the current parametrization approach, and we are actually listing each case in get_expected_code_exception anyway, I think we could change the parametrization to:

@pytest.mark.parametrize("section_kind,section_test,test_position,expected_bytecode,expected_exception",
    [
        pytest.param(
            SectionKind.TYPE,
            SectionTest.MISSING,
            CasePosition.HEADER,
            "ef000102000100030400010000800001305000ef",
            EOFException.MISSING_TYPE_HEADER,
        ),

@winsvega
Copy link
Collaborator Author

winsvega commented Jun 7, 2024

Another comment is that, since there are some repeated cases with the current parametrization approach, and we are actually listing each case in get_expected_code_exception anyway, I think we could change the parametrization to:

@pytest.mark.parametrize("section_kind,section_test,test_position,expected_bytecode,expected_exception",
    [
        pytest.param(
            SectionKind.TYPE,
            SectionTest.MISSING,
            CasePosition.HEADER,
            "ef000102000100030400010000800001305000ef",
            EOFException.MISSING_TYPE_HEADER,
        ),

hm I was thinking what if we are to add more combinations.
the current way it will be clear what are the new combinations would be. (like if I add subcontainer)
the code verification is becauses eof is in development I don't feel safe without it. (thus python falls back to json tests)

@marioevz
Copy link
Member

marioevz commented Jun 7, 2024

Another comment is that, since there are some repeated cases with the current parametrization approach, and we are actually listing each case in get_expected_code_exception anyway, I think we could change the parametrization to:

@pytest.mark.parametrize("section_kind,section_test,test_position,expected_bytecode,expected_exception",
    [
        pytest.param(
            SectionKind.TYPE,
            SectionTest.MISSING,
            CasePosition.HEADER,
            "ef000102000100030400010000800001305000ef",
            EOFException.MISSING_TYPE_HEADER,
        ),

hm I was thinking what if we are to add more combinations. the current way it will be clear what are the new combinations would be. (like if I add subcontainer) the code verification is becauses eof is in development I don't feel safe without it. (thus python falls back to json tests)

Code verification is totally fine, I like that we are not blindly trusting our EOF bytecode constructor.

What I like about this second approach is that it's easier to read for someone that comes into the file and does not have that much knowledge about how parametrization combinations work, even if we add more cases, I think it's a bit better to be explicit IMO.

@winsvega
Copy link
Collaborator Author

Another comment is that, since there are some repeated cases with the current parametrization approach, and we are actually listing each case in get_expected_code_exception anyway, I think we could change the parametrization to:

@pytest.mark.parametrize("section_kind,section_test,test_position,expected_bytecode,expected_exception",
    [
        pytest.param(
            SectionKind.TYPE,
            SectionTest.MISSING,
            CasePosition.HEADER,
            "ef000102000100030400010000800001305000ef",
            EOFException.MISSING_TYPE_HEADER,
        ),

hm I was thinking what if we are to add more combinations. the current way it will be clear what are the new combinations would be. (like if I add subcontainer) the code verification is becauses eof is in development I don't feel safe without it. (thus python falls back to json tests)

Code verification is totally fine, I like that we are not blindly trusting our EOF bytecode constructor.

What I like about this second approach is that it's easier to read for someone that comes into the file and does not have that much knowledge about how parametrization combinations work, even if we add more cases, I think it's a bit better to be explicit IMO.

what happend. now you prove me that the way json fillers expect sections were organized is better, and I prove you that python scripting is superior )

@marioevz
Copy link
Member

Another comment is that, since there are some repeated cases with the current parametrization approach, and we are actually listing each case in get_expected_code_exception anyway, I think we could change the parametrization to:

@pytest.mark.parametrize("section_kind,section_test,test_position,expected_bytecode,expected_exception",
    [
        pytest.param(
            SectionKind.TYPE,
            SectionTest.MISSING,
            CasePosition.HEADER,
            "ef000102000100030400010000800001305000ef",
            EOFException.MISSING_TYPE_HEADER,
        ),

hm I was thinking what if we are to add more combinations. the current way it will be clear what are the new combinations would be. (like if I add subcontainer) the code verification is becauses eof is in development I don't feel safe without it. (thus python falls back to json tests)

Code verification is totally fine, I like that we are not blindly trusting our EOF bytecode constructor.
What I like about this second approach is that it's easier to read for someone that comes into the file and does not have that much knowledge about how parametrization combinations work, even if we add more cases, I think it's a bit better to be explicit IMO.

what happend. now you prove me that the way json fillers expect sections were organized is better, and I prove you that python scripting is superior )

Lol, I guess I just felt it was confusing in this context, because we still need a switch-case implemented to make it work.

I think having combinations handled by parametrize is still possible, it's just that we need to figure out the logic to fill the exceptions maybe?

Copy link
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@winsvega winsvega merged commit 11ef63b into main Jun 12, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:pytest Scope: Pytest plugins
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants