Ability to log

[ctapobep]

I'd like to be able to log stuff (e.g. the $output, $status) in order to get a grip on what causes the test to fail.

[sstephenson]

Anything you print to stdout (e.g. by using echo or printf) inside your test case will be displayed if that test case fails.

Example:

#!/usr/bin/env bats

@test "output is hidden during a passing test" {
  run ls
  echo "output: $output"
  [ $status -eq 0 ]
}

@test "output is displayed during a failing test" {
  run ls nonexistent
  echo "output: $output"
  [ $status -eq 0 ]
}

 ✓ output is hidden during a passing test
 ✗ output is displayed during a failing test
   (in test file /Users/sam/Projects/bats/log.bats, line 10)
   output: ls: nonexistent: No such file or directory

2 tests, 1 failure

[brikis98]

Is there a way to see log output when tests don't fail? For example, I have some tests that run slowly and I'd like to be able to output some sort of visual indication to stdout that something is actually happening...

[ztombol]

@brikis98 This issue was opened to "display details once a test fails", which has been answered and since became easier with the bats-* libraries.

You are asking a different, but not less important, question that would have, in itself, worth opening a separate issue. Others may have a better answer, so consider opening a separate issue for visibility.

To answer your question. No, Bats only produces output once a test has exited, successfully or otherwise. See a possible workaround below.

Workaround

To display information while a test is executing, for example to monitor a long running test, one can write to a log file and tail -f it.

#!/usr/bin/env bats

log() {
  local -r log="log-${BATS_TEST_FILENAME##*/}"
  echo "$@" >> "$log"
}

@test 'long running test' {
  log "Starting test #${BATS_TEST_NUMBER}"

  # step 1
  log '  Step 1: Starting...'
  sleep 5
  log '  Step 1: Finished!'

  # step 2
  log '  Step 2: Starting...'
  sleep 5
  log '  Step 2: Finished!'

  log "Finished test #${BATS_TEST_NUMBER}"
}

Start the test as usual.

$ bats test.bats

Then, in another terminal, tail the log file.

$ tail -f -log-test.bats

And you can see how your tests are progressesing.

Starting test #1
  Step 1: Starting...
  Step 1: Finished!
  Step 2: Starting...
  Step 2: Finished!
Finished test #1

Note: Removed non-POSIX tail options (--retry, --follow and -F). -- 2016-05-10 18:31 UTC

[brikis98]

@ztombol The issue simply says "ability to log"; the "on failure" aspect was added only as part of the response.

At any rate, thanks for the workaround. I've been doing basically that already, but having to manage and tail extra files is a bit tedious. It would be great to have a flag like --verbose or --debug that shows stdout/stderr in the terminal during test execution. If it helps to have a separate issue for that, I'm happy to open one.

[ztombol]

Yeah, I know the workaround is tedious. This feature would be useful. If I'm correct, this is similar what have been asked in (#96) and proposed in (#149). If you think your suggestion is different, I think it worth opening a new issue.

I'm currently going over the issue tracker looking for features and issues that could be addressed in the next release (being organised in #150). This would be a useful feature, so I'm adding it to the list!

[brikis98]

Ah, #149 would be perfect! I'm very happy to hear it's on the list of possibilities for the next release. Thanks!