Skip to content

Latest commit

 

History

History
202 lines (161 loc) · 8.64 KB

test.org

File metadata and controls

202 lines (161 loc) · 8.64 KB

Test framework for Elvish

A very simplistic test framework for Elvish.

This file is written in literate programming style, to make it easy to explain. See test.elv for the generated file.

Table of Contents

Usage

Install the elvish-modules package using epm:

use epm
epm:install github.com/zzamboni/elvish-modules

In your rc.elv, load this module:

use github.com/zzamboni/elvish-modules/test

Writing tests

The basic block is the test:check function. It takes a lambda and an optional description, and returns a function which executes the lambda, and reports the test as passed if it returns a booleanly true value. The description, if given, is printed together with the results. All test functions return $true if all the tests it contains passed, $false otherwise.

Note that test:check returns a function, so to actually run the test, you need to execute its result by enclosing the call in parenthesis (this is so that test functions can be aggregated using test:set, see below).

~> (test:check { eq ( + 2 2 ) (float64 4) } True arithmetic)
PASS True arithmetic { eq ( + 2 2 ) (float64 4) }
~> (test:check { eq ( + 2 2 ) (float64 5) } Funky arithmetic)
FAIL Funky arithmetic { eq ( + 2 2 ) (float64 5) }
▶ $false

test:is and test:is-not are frontends to test:check which explicitly take a lambda and a value. The test passes depending on whether the result from the lambda is equal/not equal to the given value. The main difference to test:check is that when the test fails, the value returned from the lambda is reported as well.

~> (test:is { + 2 2 } (float64 4) True arithmetic)
PASS True arithmetic (eq ( + 2 2 ) 4)
▶ $true
~> (test:is { + 2 2 } (float64 5) Funky arithmetic)
FAIL Funky arithmetic (eq ( + 2 2 ) 5)
  actual: (not (eq 4 5))
▶ $false

Test functions generated with test:check, test:is and test:is-not can be grouped using test:set, which takes an identifier and a list of test functions, which get executed in sequence. The list can also contain other test:set objects. Each nested test:set counts as a single test to its outer set, but it reports its own results.

~> use github.com/zzamboni/elvish-modules/util
~> (test:set github.com/zzamboni/elvish-modules/util [
    (test:set dotify-string [
      (test:is { util:dotify-string "somelongstring" 5 } "somel…" Long string gets dotified)
      (test:is { util:dotify-string "short" 5 } "short" Equal-as-limit string stays the same)
      (test:is { util:dotify-string "bah" 5 } "bah" Short string stays the same)])])
 Testing github.com/zzamboni/elvish-modules/util
   Testing github.com/zzamboni/elvish-modules/util dotify-string
     PASS Long string gets dotified (eq ( util:dotify-string "somelongstring" 5 ) somel…)
     PASS Equal-as-limit string stays the same (eq ( util:dotify-string "short" 5 ) short)
     PASS Short string stays the same (eq ( util:dotify-string "bah" 5 ) bah)
   github.com/zzamboni/elvish-modules/util dotify-string results: 3/3 passed
 github.com/zzamboni/elvish-modules/util results: 1/1 passed
▶ $true

You can change the colors for successful and failed tests by assigning the corresponding values to $test:pass-style (default green) and $test:fail-style (default red) respectively. You can also change the string used to indent each level of nested test:set (default ' ', two spaces) by changing $test:set-indent.

Implementation

Load libraries.

use str

Configuration variables

Style to use for success/failure messages, and for other informational strings.

var pass-style = green
var fail-style = red
var info-style = blue

Indentation to use for each level of nested test:set.

var set-indent = '  '

Utility functions

Return the given text in the corresponding style according to the test result. Takes a value which will be converted to boolean using bool, and two strings, the first one for success and the second one for failure (when $result is a true or false, respectively). If the second string is empty, the first one is used in both cases, with only the style changing.

fn status {|result text-pass text-fail|
  if (eq $text-fail '') {
    set text-fail = $text-pass
  }
  var style = [&$true=$pass-style &$false=$fail-style]
  var texts = [&$true=$text-pass  &$false=$text-fail]
  var index = (bool $result)
  styled $texts[$index] $style[$index]
}

Some utility functions to produce the test output.

fn -level-indent {|level|
  repeat $level $set-indent
}

fn -output {|@msg &level=0|
  print (-level-indent $level) >/dev/tty
  echo $@msg >/dev/tty
}

Test functions

test:check is the basic building block. It takes a lamda, and returns a function which verifies that the output of the lambda is true. Optionally a description of the test can be passed, which gets printed together with the result of the test. The &check-txt option is mainly for internal use by test:is/is-not, and allows to specify the code to be displayed as the check, which by default is the source code definition of $f. The returned function runs the lambda, prints the result, and returns $true or $false depending on the result of the check. The returned function takes an option &top-id to be consistent with the API of the function returned by test:set, but this option is not used. The &level option is used to determine the indentation to use for the report.

fn check {|f @d &check-txt=''|
  var msg = (styled (str:join " " [$@d]) $info-style)
  if (eq $check-txt '') {
    set check-txt = $f[def]
  }
  put {|&top-id='' &level=0|
    var res = (bool ($f))
    -output &level=$level (status $res PASS FAIL) $msg $check-txt
    put $res
  }
}

test:compare uses test:check in the backend, but allows separate specification of the lambda to run and the value to which its output should be compared using the given $cmpfn function (its name should get passed as $cmp for the report. The advantage over test:check is that it can report not only whether the check failed, but also which value was produced instead of the expected one.

fn compare {|cmp cmpfn f v @d|
  put {|&top-id='' &level=0|
    var res = ($f)
    var check-res = ((check { $cmpfn $res $v } $@d &check-txt='('$cmp' ('$f[body]') '(to-string $v)')') &level=$level)
    if (not $check-res) {
      -output &level=$level "  actual: (not ("$cmp' '(to-string $res)' '(to-string $v)'))'
    }
    put $check-res
  }
}

test:is and test:is-not are shortcuts for test:compare with eq and not-eq as comparison functions, respectively.

fn is {|f v @d|
  compare eq $eq~ $f $v $@d
}
fn is-not {|f v @d|
  compare not-eq $not-eq~ $f $v $@d
}

test:set receives a description and an array containing test functions (can be other test:set’s, test:check, test:compare, test:is or test:is-not) and returns a function which calls them in sequence. A header with the description is printed. In nested test:set objects, the description of the enclosing set gets prepended, separated by a space. It keeps count of how many of the checks succeed, and prints a report at the end. It returns $true if all the checks passed, $false otherwise.

fn set {|id tests|
  put {|&top-id="" &level=0|
    if (not-eq $top-id '') {
      set id = $top-id' '$id
    }
    -output &level=$level (styled "Testing "$id $info-style)
    var -nextlevel = (+ $level 1)
    var passed = (each {|t|
        if ($t &top-id=$id &level=$-nextlevel) { put $true }
    } $tests | count)
    var res = (eq $passed (count $tests))
    var msg = (status $res $passed"/"(count $tests)" passed" '')
    -output &level=$level (styled $id" results:" $info-style) $msg
    put $res
  }
}