Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Housekeeping] Support pytest-xdist #4446

Closed
2 tasks done
thomasjpfan opened this issue Nov 17, 2023 · 1 comment · Fixed by flyteorg/flytekit#1974
Closed
2 tasks done

[Housekeeping] Support pytest-xdist #4446

thomasjpfan opened this issue Nov 17, 2023 · 1 comment · Fixed by flyteorg/flytekit#1974
Labels
flytekit FlyteKit Python related issue housekeeping Issues that help maintain flyte and keep it tech-debt free

Comments

@thomasjpfan
Copy link
Member

Describe the issue

pytest-xdist allows us to run the tests in parallel, which speeds up local development and the CI. GitHub actions allocates 2 cores per runner. With pytest -n I get the follow runtimes locally:

  • 1 core (default): 121 seconds
  • 2 cores: 80 seconds
  • 4 cores: 48 seconds

The issue is that 4 tests fail. If we adjusts these tests, then we can get pytest-xdist working.

FAILED tests/flytekit/unit/core/test_conditions.py::test_condition_tuple_branches - AssertionError: assert '__channelexe...tions.sum_sub' == 'tests.flytek...tions.sum_sub'
FAILED tests/flytekit/unit/core/test_type_hints.py::test_error_messages - AssertionError: Regex pattern did not match.
FAILED tests/flytekit/unit/core/test_type_hints.py::test_union_type - AssertionError: Regex pattern did not match.
FAILED tests/flytekit/unit/core/test_workflows.py::test_wf_nested_comp - AssertionError: assert '__channelexe..._workflows.t1' == 'tests.flytek..._workflows.t1'

What if we do not do this?

We would not be taking advantage of all the CI compute and local compute for development.

Related component(s)

flytekit

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@thomasjpfan thomasjpfan added the housekeeping Issues that help maintain flyte and keep it tech-debt free label Nov 17, 2023
Copy link

welcome bot commented Nov 17, 2023

Thank you for opening your first issue here! 🛠

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flytekit FlyteKit Python related issue housekeeping Issues that help maintain flyte and keep it tech-debt free
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant