Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avocado instrumented timeout in setUp and tearDown error #6013

Closed
richtja opened this issue Aug 28, 2024 · 3 comments
Closed

Avocado instrumented timeout in setUp and tearDown error #6013

richtja opened this issue Aug 28, 2024 · 3 comments
Assignees
Labels

Comments

@richtja
Copy link
Contributor

richtja commented Aug 28, 2024

Describe the bug
A clear and concise description of what the bug is.
When the test timeout is reached during setUp or tearDown methods, the test results as ERROR instead of INTERUPTED. Based on my testing, the interruption is handled properly, only the result value is wrong.

Steps to reproduce
Steps to reproduce the behavior with a reproducible whenever possible.
Test setUp example:

import time

from avocado import Test


class TimeoutTest(Test):
    """
    Functional test for avocado. Throw a TestTimeoutError.

    :param sleep_time: How long should the test sleep
    """

    timeout = 3

    def setUp(self):
        time.sleep(5)

    def test(self):
        """
        This should throw a TestTimeoutError.
        """
        pass

Test tearDown example:

import time

from avocado import Test


class TimeoutTest(Test):
    """
    Functional test for avocado. Throw a TestTimeoutError.

    :param sleep_time: How long should the test sleep
    """

    timeout = 3

    def test(self):
        """
        This should throw a TestTimeoutError.
        """
        pass

    def tearDown(self):
        time.sleep(5)

Expected behavior
A clear and concise description of what you expected to happen.

$ avocado run test.py
JOB ID     : d3e279280bb6666bc0c4b05fdcbd5d8b2e4ff135
JOB LOG    : /home/janrichter/avocado/job-results/job-2024-08-28T10.12-d3e2792/job.log
 (1/1) test.py:TimeoutTest.test: STARTED
 (1/1) test.py:TimeoutTest.test: INTERRUPTED: Test interrupted: Timeout reached (3.07 s)
RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 1 | CANCEL 0
JOB HTML   : /home/janrichter/avocado/job-results/job-2024-08-28T10.12-d3e2792/results.html
JOB TIME   : 4.79 s

Current behavior
What you are getting?

$ avocado run test.py
JOB ID     : f11768c3dfd8f2356fdd94b79300a84c167ac717
JOB LOG    : /home/janrichter/avocado/job-results/job-2024-08-28T10.10-f11768c/job.log
 (1/1) test.py:TimeoutTest.test: STARTED
 (1/1) test.py:TimeoutTest.test: ERROR: Test interrupted: Timeout reached (3.01 s)
RESULTS    : PASS 0 | ERROR 1 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /home/janrichter/avocado/job-results/job-2024-08-28T10.10-f11768c/results.html
JOB TIME   : 4.70 s

System information (please complete the following information):

  • OS:
LSB Version:    :core-5.0-amd64:core-5.0-noarch
Distributor ID: Fedora
Description:    Fedora release 40 (Forty)
Release:        40
Codename:       Forty

  • Avocado version: master
  • Avocado installation method: github
@richtja richtja added the bug label Aug 28, 2024
@clebergnu
Copy link
Contributor

Hi @richtja, thanks for opening this issue. I think the main point this issue proposes is valid, that is, if setUp times out, and gets interrupted, it should have an INTERRUPTED status.

BTW, IMO this issue should cover fixing the behavior, but also documenting the proposed timeout behavior. When issue #5994 is implemented, then the documentation should be updated to cover the individual timeouts for the different phases.

@richtja
Copy link
Contributor Author

richtja commented Aug 28, 2024

Hi @richtja, thanks for opening this issue. I think the main point this issue proposes is valid, that is, if setUp times out, and gets interrupted, it should have an INTERRUPTED status.

BTW, IMO this issue should cover fixing the behavior, but also documenting the proposed timeout behavior. When issue #5994 is implemented, then the documentation should be updated to cover the individual timeouts for the different phases.

IMO the current timeout behaviour is documented by #6012 and if we agree on that any timeout should have INTERRUPTED status than I think we don't need to add more documentation to this. Of course, the #5994 should also have documentation part of that new feature.

@richtja richtja self-assigned this Aug 29, 2024
richtja added a commit to richtja/avocado that referenced this issue Aug 29, 2024
This commit is a fix for avocado-instrumented timeouts. When the tests
are interrupted due to timeout during `setUp` or `tearDown` method,
those tests would result in `ERROR` instead of `INTERRUPTED`. This
commit updates the exception handling of this method to fix this issue.

Reference: avocado-framework#6013
Signed-off-by: Jan Richter <[email protected]>
richtja added a commit to richtja/avocado that referenced this issue Sep 16, 2024
This commit is a fix for avocado-instrumented timeouts. When the tests
are interrupted due to timeout during `setUp` or `tearDown` method,
those tests would result in `ERROR` instead of `INTERRUPTED`. This
commit updates the exception handling of this method to fix this issue.

Reference: avocado-framework#6013
Signed-off-by: Jan Richter <[email protected]>
richtja added a commit to richtja/avocado that referenced this issue Sep 16, 2024
This commit is a fix for avocado-instrumented timeouts. When the tests
are interrupted due to timeout during `setUp` or `tearDown` method,
those tests would result in `ERROR` instead of `INTERRUPTED`. This
commit updates the exception handling of this method to fix this issue.

Reference: avocado-framework#6013
Signed-off-by: Jan Richter <[email protected]>
richtja added a commit to richtja/avocado that referenced this issue Sep 16, 2024
This commit is a fix for avocado-instrumented timeouts. When the tests
are interrupted due to timeout during `setUp` or `tearDown` method,
those tests would result in `ERROR` instead of `INTERRUPTED`. This
commit updates the exception handling of this method to fix this issue.

Reference: avocado-framework#6013
Signed-off-by: Jan Richter <[email protected]>
@richtja
Copy link
Contributor Author

richtja commented Sep 18, 2024

Resolved by #6015

@richtja richtja closed this as completed Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done 108
Development

No branches or pull requests

2 participants