-
Notifications
You must be signed in to change notification settings - Fork 517
Stuck on the last test of each file #80
Comments
Could you isolate the failing test in a short example? I got some hangs too, not reproducible yet, it may be related to IFS see #89. could you give your bash version? |
Unfortunately, I can't isolate any specific test failing. The only thing common to all the hang ups is that they always happen on the last test of the first test file evaluated. After I read it, I'm not so sure it is related to #89 (I actually don't know what this IFS is neither) |
$IFS is internal bash variable, used in auto spliting, I've pushed on my forked version and preparing the pull request. from man bash:
|
Thanks for the info :-) Gonna try with your fork see if it fixes things, never know... |
I might have the same issue: In one of my tests I call an This is a simple way to reproduce it: #!/usr/bin/env bats
@test "fork something" {
sleep 5 &
echo 'forked'
}
@test "hanging check" {
echo 'hang'
} This stalls bats until the 5s are over. This version may come closer to my init-script case: #!/usr/bin/env bats
@test "fork something" {
(while true; do sleep 1; done) &
echo 'forked'
}
@test "hanging check" {
echo 'hang'
} You have to kill the wait with ctrl-c, then the check mark appears next to "hanging check" and bats terminates. I see the same behaviour in master-branch and in tags/v0.4.0. I'm new to bats. thus is blind guessing: Does bats look after forked processes and waits for them to terminate? |
In my init-script example I can terminate the hang by stopping the started daemon from another shell. Furthermore, I have found a race condition in my test. One check has started the Apache httpd and another was supposed to stop it. Without anything else to test and do between start and stop, I believe the stop-test to be triggered before the forked httpd was ready to accept shutdown signals. With more tests and some tactical @oleiade, do you have any "asymmetric" forks in your test, too? |
@filex why not waiting for the pid? maybe starting the process with nohup or something similar writing the pid into a file. You don't start a test in a one test, and check the result in another, aren't you? I think you have to detach the process looping forever in order do finish the test. |
Yes, @Sylvain303, I start a daemon, run several checks and stop it lastly. This is split up in several tests to have somewhat meaningful test names for everything. My problem was the race condition between start and stop. Everything is working fine now. But apparently When I call an init script (apache and my own software) in a shell, the processes are forked and vanish from the shells However, |
hum, no idea for the moment. I stepped in bats code for a bug, it parses the .bats, generate a valid shell script in a tmp and execute each function, I gonna try your bug to see I can see where it hangs. I will probably to have to write similar tests. But dont expect a fast reply from me ;)… |
I ran into this and tried nohup, &, pid catching, all of those. |
It's not stuck on the last test, but after it... The problem is that bats uses file descriptor 3 as a communication channel among its internal processes. Your background sleep process inherits FD#3 from bats, as the write end of a pipe. Since your process is holding FD#3 open, the pipe doesn't report EOF to its reader, and bats, which is waiting for that to happen, hangs. When the sleep process exits, its FD#3 gets closed, and since that was the last writer, the pipe finally reports EOF. I've tried to come up with a patch for bats, but haven't suceeded. I'd have expected this to work:
but it doesn't; with it, "pretty output" only contains the checkmarks, no test names! I don't know why. As a workaround, try changing the sleep in your test to:
|
Closes #226. From the comment within `run_in_background`: Bats duplicates standard output as file descriptor 3 so that output from its framework functions isn't captured along with any output from the code under test. If the code under test contains a `sleep` or other blocking operation, this file descriptor will be held open until the process becomes unblocked, preventing Bats from exiting. Hence, we explicitly close file descriptor 3. Any other code running under Bats that opens a background process should close this file descriptor as well. See: sstephenson/bats#80 Much thanks to @marascio for discovering and researching the problem, and proposing the actual fix.
Closes sstephenson#72 and sstephenson#81. Both of those issues deal with the `bats_error_trap` executing with an incorrect value for `$?`. Bats exited with an error, but provided no stack trace information to pinpoint its source. * sstephenson#72 deals with the inconsistent failure-mode behavior of the `source` builtin between Bash versions. I believe it's feasible to update `load` with robust error checking around `source` and strongly recommend its use in Bats test files instead of using `source` directly. I've already opened sstephenson#80 to track this. * sstephenson#81 deals with inconsistent failure-mode behavior of `set -u`, particularly when code within a Bats test file itself references an unbound variable. With credit to sstephenson, Bats was smart enough to know and to report that an error occurred, even when it couldn't tell exactly where the error came from. Building on that existing mechanism, this change produces output for these failure cases even when `bats_error_trap` isn't called with the correct value for `$?`. It passes all the existing tests under Bash 3.2.57(1)-release and 4.4.19(1)-release. I also timed the original suites with and without the change to ensure the runtime cost was negligible. A couple more notes: * This change obsoletes the suggestion in sstephenson#81 that a test case to validate testing code running under `set -u` is necessary. `run()` disables `set -e` for the code under test to begin with, sidestepping the problem with `set -eu` interactions in Bash versions prior to 4.1.0. * I've noticed that this mechanism does _not_ work when the problematic behavior is in `teardown()`. The change preserves existing behavior, the remedy for that issue requires further thinking (and a new issue).
@esiegerman Thanks for your workaround. Can you please explain what |
Hi,
I'm extensively using the bats testing framework in trousseau. I have written a bunch of tests. Everything works fine on my laptop (OSX), and in vagrant boxes (Ubuntu 14.04), however when I run the tests in a CI environment like codeship or travis I noticed that bats always end up stuck on the last test of each files.
For example:
Any ideas what could cause this? a wait4 syscall never ending? A SIGCHLD never received? An improper redirection of stdout/stderr hiding the real problem?
Thanks for your help :-)
Theo
The text was updated successfully, but these errors were encountered: