-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement true tests #22
Comments
Do you think we can get this fixed before SfN? Weren't you working on exactly this kind of thing with the OSM fork? |
Well, the OSB model validator is mainly concerned with testing models, not the simulators themselves. |
I was just looking through old notes from 2.3 beta testing, and trying to recover what we did. There was a genesis/tests/TestSuite directory that was not included in the final release. I'll come up with some tests to recommend later today. |
I looked at the old TestSuite, and don't think it is worth reimplementing now. Here are some simple Suggested tests for accuracy; (1) Test the Scalable Portable Random Number Generator (SPRNG) genesis #1 > setrand -sprng Regardless of platform, you should get the results: 0.01426654216 0.7493918538 0.007316101808 0.1527428776 0.1134621128 (2) Test the rallpacks 'axon.g' simulation: cd rallpack/reports/genesis-2.0/rallpack3 run axon.g, which will produce the files axon.out0 and axon.outx The last entries should be: $ tail -n 5 axon.outx I get the same results for both genesis executables, but the one produced by configure takes 0.351000 cpu seconds, and the one from the edited Makefile.dist takes 0.267000 cpu seconds We should look at the optimization flags |
I've adapted (f76929a) the travis test to compare the output of sprng to the expected values mentioned above. The builds will fail if the comparison fails, so we now have a better indicator (well, at least better than just checking if make is returning 0) of the validity of the binaries.
|
Do you get the same results with both methods? As you know, round off errors are tricky when The configure script is not using the '-O2' option, and it links -lSm and -lICE, which are not needed |
The configure script now uses '-O2' as of af6b988, libSM and libICE were also removed in the same build. My output is shifted by 0.00005 (seconds?):
and here it simply differs slightly:
What does that mean? |
I tried this on a colleague's Linux machine (recent Ubuntu x86_64) and it produces the same numbers as in my case. Any ideas why? |
We need to have a systematic way of testing the builds for correctness. Rallpack?
The travis script (#21) needs to be adapted to run such tests.
The text was updated successfully, but these errors were encountered: