This is a test suite for the compiler, simulator, and tools for the Bluespec Hardware Description Language as found in the bsc repository. Some testing is also available for libraries in the bsc-contrib repository.
This test suite uses the DejaGnu testing framework, which is written in Expect, which uses Tcl. The following command will install those packages on Debian or Ubuntu:
$ apt-get install dejagnu
The test suite also uses a number of other tools to run and test programs:
$ apt-get install csh grep m4 make perl pkg-config time
To simulate the Verilog generated by BSC, a Verilog simulator is required. By default, the test suite expects Icarus Verilog, which can be installed on Debian or Ubuntu with the following command:
$ apt-get install iverilog
When a Verilog simulator is not available, simulation tests can be disabled
by assigning VTEST=0 in the environment or on the command line when running
the make commands shown below.
Tests for BSC's SystemC generation require SystemC headers and libraries for compilation and linking.
SystemC can be installed on Debian 10 (buster) or later with the following command:
$ apt-get install libsystemc-dev
When SystemC is not available, these tests can be disabled by assigning
SYSTEMCTEST=0 in the environment or on the command line when running
the make commands shown below. Debian 10 (buster) and later have
Clone this repository by running:
$ git clone https://github.com/B-Lang-org/bsc-testsuite
Running tests in individual directories uses git to find the top
directory of the testsuite, so it is best to leave the test suite as a
git repository. If the suite is exported or archived outside of a git
repository, some features may not work.
There are many ways to run tests in the suite, but the simplest is:
$ make TEST_RELEASE=/path/to/bsc/inst check
This will run the suite on the BSC installation pointed to by TEST_RELEASE.
Actually, an even simpler command is possible. If you place a bsc
repository as a sibling directory to the test suite repository, and it
has an inst subdirectory, the Makefile can detect that and
implicitly assign TEST_RELEASE if you have omitted it:
$ make check
If you omit TEST_RELEASE and there is no sibling bsc with an inst
subdirectory, the Makefile will report an error.
Some tests will only run if the bsc-contrib libraries have been
installed. You can specify a location where the files are
installed and that will enable these tests:
$ make TEST_CONTRIB=/path/to/bsc-contrib/inst check
As with TEST_RELEASE, you can omit this assignment and the Makefile
will implicitly assign TEST_CONTRIB if either the BSC repository
that you are testing or the test suite repository that you are running
from has a sibling directory named bsc-contrib with an inst
subdirectory. If you omit TEST_CONTRIB and there is no sibling
bsc-contrib with an inst subdirectory, the Makefile will disable
these tests.
By default, the Makefile in bsc/src/comp/ will not build and install
the tool showrules or the developer tools. These tools can all be
installed with the install-extra target in bsc/src/comp/ (or
separately with install-showrules and install-utils).
The test has tests for the showrules tool in bsc.showrules.
These tests are disabled if showrules is not found in the
TEST_RELEASE bin directory.
The test suite is currently able to use the following developer
tools, if they exist in the TEST_RELEASE bin directory:
dumpbo, vcdcheck, and bsc2bsv. There is also a dumpba
tool, which is not being used by the test suite, but could be
usefully added.
After each compilation that generates a .bo files, the suite
can perform a sanity check by running dumpbo on the file
and checking for an error exit code. There are also a small
number of tests that explicitly run dumpbo as part of their
testing.
The suite does not currently perform a sanity check on
generated .ba files, but that would be possible with
dumpba as a future extension.
After each Bluesim simulation that generates a .vcd file,
the suite can perform a sanity check by running vcdcheck
on the file and checking for an error exit code. There are
also a small number of tests that explicitly run vcdcheck
as part of their testing.
One test (bsc.bugs/b611/) checks for a bug in bsc2bsv
but otherwise this tool is not used. There are no tests
for the related tool, bsv2bsc.
If you want to perform a quick check, the smoke target will
run a small subset of tests:
$ make smoke
The set of tests that are run is specified in Makefile.
On a clean repository, the check target runs most tests,
but not all. Several tests have been deemed to be long
tests and not essential to run. These have been placed
in bsc.long_tests/ and they have been disabled. There
is a Makefile in that directory which has targets for
enabling and disabling tests, as well as showing the
currently enabled tests. At the top tevel of the suite,
the fullcheck command will enable these tests and then
run check:
$ make fullcheck
After that command has been run, making check will run
all the tests, so remember to disable the tests afterward
if you don't want to run them (or use git's clean command
to restore the repo).
If you want to run only the tests in a given subdirectory, you can change to that directory and do:
$ make localcheck
The localcheck target will only run the tests in the
current directory.
If you want to test the current directory and recurse into its subdirectories, you can do:
$ make check
This will run tests in the current directory and print a report and then, if there are no failures in that report, it will immediately run the tests in the subdirectories and print a report for those. (So be aware that there will be two reports, if you want to see all the stats.)
There are also checkparallel and fullparallel targets,
which are versions of check and fullcheck that run the
tests in parallel.
NOTE: These targets may have issues to be fixed.
If you do not want to run Bluesim simulations, you can disable
those tests (or parts of tests) by assigning CTEST=0.
For example:
$ make CTEST=0 check
If you do not want to run Verilog simulations, you can disable
those tests (or parts of tests) by assigning VTEST=0.
For example:
$ make VTEST=0 check
If you do not want to run tests that require SystemC, you can
disable those tests by assigning SYSTEMCTEST=0
For example:
$ make SYSTEMCTEST=0 check
BSC can be used to link Verilog files into a simulation executable.
When run this way, the choice of Verilog simulator is specified
with the -vsim flag. For example, to specify Icarus Verilog as
the simulator, the flag would be -vsim iverilog.
This is how all Verilog simulation is done in the test suite.
The argument to -vsim is provided with the TEST_BSC_VERILOG_SIM
environment variable:
$ make TEST_BSC_VERILOG_SIM=cvc64 check
The default value is iverilog.
TBD
This test suite and most tests are provided by Bluespec Inc and
available under the BSD-3-Clause license. Some test directories
contain files with copyrights to other authors, some under other
open source licenses. See COPYING for copyright and license details.