feat(anta): Limit concurrency #680
Conversation
carl-baillargeon
force-pushed
the
refactor/runner_limit
branch
from
bf88158 to
d7de654
Compare
|
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
carl-baillargeon
force-pushed
the
refactor/runner_limit
branch
from
b623e90 to
4879120
Compare
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
mtache
changed the title
mtache
force-pushed
the
refactor/runner_limit
branch
from
68e259e to
83206e8
Compare
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
CodSpeed Performance ReportMerging #680 will not alter performanceComparing Summary
Benchmarks breakdown
|
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
anta/tools.py
Outdated
| while pending or not coros_ended: | ||
| while len(pending) < limit and not coros_ended: | ||
| try: | ||
| # NOTE: The `anext` built-in function is not available in Python 3.9 | ||
| coro = await coroutines.__anext__() # pylint: disable=unnecessary-dunder-call | ||
| except StopAsyncIteration: # noqa: PERF203 | ||
| coros_ended = True | ||
| else: | ||
| pending.add(asyncio.create_task(coro)) | ||
|
|
||
| if not pending: | ||
| return | ||
|
|
||
| done, pending = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED) | ||
|
|
||
| while done: | ||
| yield done.pop() |
There was a problem hiding this comment.
| while pending or not coros_ended: | |
| while len(pending) < limit and not coros_ended: | |
| try: | |
| # NOTE: The `anext` built-in function is not available in Python 3.9 | |
| coro = await coroutines.__anext__() # pylint: disable=unnecessary-dunder-call | |
| except StopAsyncIteration: # noqa: PERF203 | |
| coros_ended = True | |
| else: | |
| pending.add(asyncio.create_task(coro)) | |
| if not pending: | |
| return | |
| done, pending = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED) | |
| while done: | |
| yield done.pop() | |
| while pending or not coros_ended: | |
| async for test in coroutines: | |
| # Add tests to the pending set until the limit is reached or no more tests are available | |
| if len(pending) < limit: | |
| # Ensure the coroutine is scheduled to run and add it to the pending set | |
| pending.add(asyncio.create_task(test)) | |
| logger.debug("Added a test to the pending set: %s", test) | |
| else: | |
| logger.debug("Concurrency limit reached: %s tests running. Waiting for tests to complete.", limit) | |
| break | |
| if len(pending) < limit and not coros_ended: | |
| coros_ended = True | |
| logger.debug("All tests have been added to the pending set.") | |
| if not pending: | |
| logger.debug("No pending tests and all tests have been processed. Exiting.") | |
| return | |
| done, pending = await asyncio.wait(pending, return_when=asyncio.FIRST_COMPLETED) | |
| while done: | |
| yield done.pop() |
|
| # AntaRunner returns test results as they finish, not necessarily in order | ||
| # TODO: Add CLI option to choose sorting order | ||
| ctx.obj["result_manager"].sort(sort_by=["name", "test", "result", "custom_field"]) |
There was a problem hiding this comment.
do we want this in this PR? (other perf impact I would think)
Description
This PR improves the test runner by introducing a generator-based approach for managing test coroutines and setting a configurable limit on the number of concurrent tests.
Instead of loading all test coroutines into a list, the runner now uses a generator to yield tests. This approach prevents memory overload and improves performance when dealing with a large number of tests.
A limit on the number of concurrent tests is introduced to avoid overwhelming the runner. This limit is configurable with an environement variable.
Implementation:
The generator yields test coroutines, ensuring that only a limited number of tests are scheduled and run concurrently.
Upon reaching the concurrency limit, the runner waits for some tests to complete before scheduling new ones from the generator.
Fixes: #832
Checklist: