-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some thoughts #43
Comments
Hi thank you for the feedback! What is a GFMD? And what should the result be?
What is the output control data? How do you compare that? A simple In seqan2 we had some magic python scripts that did the testing, now we said that C++ is the main source for the tests (as the app itself will be written in C++). I agree that the flexibility might be overwhelming when you start and if we can streamline the simple use cases that it would be awesome. Do you have any examples in seqan2 that are simple tests? But looking at seqan2 apps, like bs_tools, I guess that a cmake command would be more complex than just 3 parameters. conf = app_tests.TestConf(
program=path_to_bisar,
redir_stdout=ph.outFile('other.stdout'),
args=['-e3', str(4), '-e4', str(5),
#-e3 4 -e4 5
'-o', ph.outFile('reads_se_N6000_0.CT_GA.verified.sam'),
ph.inFile('reads_se_N6000.CT_GA.sam'),
ph.inFile('hg18_chr21_3000.fa'),
ph.inFile('reads_se_N6000.fastq')],
to_diff=[#(ph.inFile('STDOUT_FILE'),
#ph.outFile('STDOUT_FILE')),
(ph.inFile('reads_se_N6000_0.CT_GA.verified.sam'),
ph.outFile('reads_se_N6000_0.CT_GA.verified.sam'),
transforms)])
conf_list.append(conf) let's figure out some cmake syntax (based on https://cmake.org/cmake/help/latest/command/execute_process.html): # ignoring the original post-processing of the data:
# some test description
cli_test (
bisar_target
-e3 4
-e4 5
-o reads_se_N6000_0.CT_GA.verified.sam
reads_se_N6000.CT_GA.sam
hg18_chr21_3000.fa.sam
reads_se_N6000.fastq
DATASOURCES
reads_se_N6000.CT_GA.sam # will be copied into current working directory
hg18_chr21_3000.fa.sam # will be copied into current working directory
reads_se_N6000.fastq # will be copied into current working directory
TEST_DATASOURCES
reads_se_N6000_0.CT_GA.verified.sam.expected # will be copied into current working directory
OUTPUT_FILE_DIFF
# convention over configuration:
# diff reads_se_N6000_0.CT_GA.verified.sam with reads_se_N6000_0.CT_GA.verified.sam.expected
reads_se_N6000_0.CT_GA.verified.sam
) compared to TEST_F(cli_test, some_test_description)
{
cli_test_result result = execute_app("bisar",
"-e3", "4", "-e4", "5",
"-o", "reads_se_N6000_0.CT_GA.verified.sam",
data("reads_se_N6000.CT_GA.sam"),
data("hg18_chr21_3000.fa.sam"),
data("reads_se_N6000.fastq"));
// simple diff
EXPECT_FILE_CONTENT_EQ("reads_se_N6000_0.CT_GA.verified.sam", data("reads_se_N6000_0.CT_GA.verified.sam.expected"));
} I think we can't hide any inherent test-declaration complexity. Neither by using bash / pyhton / cmake or C++. I generally would argue that staying within one echo system is preferable. google-test is known to most of us. |
We want to offer a github-action for that. Creating a new PR if a new release candidate / stable release is available. |
Github-flavored Markdown! The result should be something you can copy and paste into your README.
I usually just compute checksums on the files.
Oh, this looks much better than I expected! I thought that users need to edit [I would maybe move the And I would put more focus on these tests and not promote the others as much. I also found the folder names in the top-level directory a bit surprising. I know that these things are subjective, but I personally would do it like this:
AFAIK
That way we have less top-lovel folders and the typical/easy things are immediately visible. People know that there is more fancy stuff ("advanced") but also know that they don't need to look at it or understand it to do the other stuff. [Prevents the "What's all of this?" -- reaction] |
This is already planned :) And I like your suggestions about the directories and CLI documentation! |
First of all: it's great to have this!
I just looked through this repository and had the following very subjective thoughts. Please see this as constructive criticism :)
There are many things I found surprising to see / have never seen with applications before. This includes most of the test things, but also Doxygen. While the terms might be clear for a library developer, in the context of the APP, they are not clear to me. Does an app have an API? And if yes, isn't this the CLI? Why are tests for the CLI written inside C++? I don't think I have ever seen that before, and I am fairly sure regular app developers have never heard of GMock etc.
Similarly: what does the documentation cover? The interface of the binary or some functions inside the app source code? Why would users of the app be interested in the latter?
Don't get me wrong, I do think that these things might make sense for large applications that have multiple developers and are maintained for a longer period of time. And I also think that it is kind of fancy that one can write app-tests from inside C++ and have coverage checked... I just don't see 90% of this happening in most academic projects or even projects in my company. So for the assumed primary audience, it feels a little over-engineered -> there are a lot of things to understand that are likely not relevant, while those things that are, are not easy to do.
Being an app-developer myself, I am primarily interested in the following:
make doc-html
,make doc-md
...make update-seqan3
updates the SeqAn3 submodule to the latest stable release and prints the release notes :)The text was updated successfully, but these errors were encountered: