Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple timeouts within a test #5994

Open
clebergnu opened this issue Aug 4, 2024 · 0 comments · May be fixed by #6031
Open

Multiple timeouts within a test #5994

clebergnu opened this issue Aug 4, 2024 · 0 comments · May be fixed by #6031

Comments

@clebergnu
Copy link
Contributor

clebergnu commented Aug 4, 2024

Is your feature request related to a problem? Please describe.
On more complex (and thus usually) longer tests, there may be multiple steps to complete. It may be known that some of these steps should not take more than a small percentage of the overall expected time for the test as whole. Example:

class MyTest(Test):

   TIMETOUT = 100

   def test(self):
      # stimulation of a few bytes of entropy should not take
      # more than 5 seconds
      generate_entropy()

      # the bulk of the work in the test is here, and can take
      # as much as 90 seconds
      verify_crypto_functions()

In the current situation, with a single TIMEOUT per test, generate_entropy() could take 20 times longer before the test is considered timed out.

Describe the solution you'd like
Support for defining timeouts for different parts of the test, such as:

class MyTest(Test):

   TIMETOUT = 100

   def test(self):
      # stimulation of a few bytes of entropy should not take
      # more than 5 seconds
      with wait_max(5):
          generate_entropy()

      # the bulk of the work in the test is here, and can take
      # as much as 90 seconds
      verify_crypto_functions()

The wait_max() context manager would raise an exception if generate_entropy() takes longer than 5 seconds. This exception should be treated by the runner so that the test ends with a proper INTERRUPTED status and a description of where it timed out.

Additional information
This feature is being proposed based on the discussions and requirements defined in https://gitlab.com/avocado-framework/dovetail/-/issues/4

@mr-avocado mr-avocado bot moved this to Triage in Default project Aug 4, 2024
@clebergnu clebergnu moved this from Triage to Short Term (Current Q) Backlog in Default project Aug 5, 2024
@richtja richtja self-assigned this Sep 12, 2024
@richtja richtja moved this from Short Term (Current Q) Backlog to In progress in Default project Sep 12, 2024
richtja added a commit to richtja/avocado that referenced this issue Sep 17, 2024
This commit introduces a new timeout feature, which adds the possibility
to set a specific timeout for a block of code with context manager like
this:

	whit wait_max(3):
		#code which should take max 3 seconds
		...

The  `wait_max` method will send `SIGALRM` if the code doesn't end
within 3 seconds. This signal will be caught by avocado-instrumented
runner, which will interrupt the test, the same way as with a regular
timeout.

Reference: avocado-framework#5994
Signed-off-by: Jan Richter <[email protected]>
richtja added a commit to richtja/avocado that referenced this issue Sep 17, 2024
This commit introduces a new timeout feature, which adds the possibility
to set a specific timeout for a block of code with context manager like
this:

	with self.wait_max(3):
		#code which should take max 3 seconds
		...

The  `wait_max` method will send `SIGALRM` if the code doesn't end
within 3 seconds. This signal will be caught by avocado-instrumented
runner, which will interrupt the test, the same way as with a regular
timeout.

Reference: avocado-framework#5994
Signed-off-by: Jan Richter <[email protected]>
@richtja richtja linked a pull request Sep 17, 2024 that will close this issue
richtja added a commit to richtja/avocado that referenced this issue Sep 18, 2024
This commit introduces a new timeout feature, which adds the possibility
to set a specific timeout for a block of code with context manager like
this:

	with self.wait_max(3):
		#code which should take max 3 seconds
		...

The  `wait_max` method will send `SIGTERM` if the code doesn't end
within 3 seconds. This signal will be caught by avocado-instrumented
runner, which will interrupt the test, the same way as with a regular
timeout.

Reference: avocado-framework#5994
Signed-off-by: Jan Richter <[email protected]>
richtja added a commit to richtja/avocado that referenced this issue Sep 18, 2024
This commit introduces a new timeout feature, which adds the possibility
to set a specific timeout for a block of code with context manager like
this:

	with self.wait_max(3):
		#code which should take max 3 seconds
		...

The  `wait_max` method will send `SIGTERM` if the code doesn't end
within 3 seconds. This signal will be caught by avocado-instrumented
runner, which will interrupt the test, the same way as with a regular
timeout.

Reference: avocado-framework#5994
Signed-off-by: Jan Richter <[email protected]>
@richtja richtja linked a pull request Sep 18, 2024 that will close this issue
richtja added a commit to richtja/avocado that referenced this issue Oct 2, 2024
This commit introduces a new timeout feature, which adds the possibility
to set a specific timeout for a block of code with context manager like
this:

	with self.wait_max(3):
		#code which should take max 3 seconds
		...

The  `wait_max` method will send `SIGTERM` if the code doesn't end
within 3 seconds. This signal will be caught by avocado-instrumented
runner, which will interrupt the test, the same way as with a regular
timeout.

Reference: avocado-framework#5994
Signed-off-by: Jan Richter <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In progress
Development

Successfully merging a pull request may close this issue.

2 participants