Integrate Googletest as Test Framework

[zhjwpku]

In the discussion #12 regarding the choice between Googletest and Catch2 for our testing framework, most of the developers have experience with Googletest, and while they are open to considering Catch2, they see no immediate advantage, as no one on the team has experience with it. Given this context, the decision to adopt Googletest was made based on the familiarity and comfort level of the majority of the team.

We can switch to Catch2 with one shot if it proves to be a better fit for our needs in the future.

[pitrou]

Thanks @zhjwpku !

What I would be interested to know is the compile and link times, respectively, for both GTest and Catch2.
My experience in Arrow is that GTest's main header gtest.h imposes a large compilation overhead when you have many independent test modules, and I'm curious if Catch2 improves the situation.

[zhjwpku]

Thanks @zhjwpku !

What I would be interested to know is the compile and link times, respectively, for both GTest and Catch2. My experience in Arrow is that GTest's main header gtest.h imposes a large compilation overhead when you have many independent test modules, and I'm curious if Catch2 improves the situation.

I did 3 tests:

1. using -E to see the preprocessed file size
   • 4301941 -- gtest
   • 4086315 -- catch2
2. using time to see the compile time
   • 0.40s user 0.06s system 78% cpu 0.587 total -- gtest
   • 0.32s user 0.06s system 76% cpu 0.494 total -- catch2
3. using time to see the link time
   • 0.04s user 0.02s system 79% cpu 0.080 total -- gtest
   • 0.09s user 0.03s system 98% cpu 0.120 total -- catch2

As the results shows, catch2 is better at preprocessed file size and compile time, but gtest is better when it comes to link time. I don't think the above tests is adequate to prove which is better than the other, since the UT in this PR is fairly simple.

I think we should focus closely on the specific benefits that the test framework is expected to deliver for this project.

[pitrou]

Thanks for the number @zhjwpku .

As the results shows, catch2 is better at preprocessed file size and compile time, but gtest is better when it comes to link time. I don't think the above tests is adequate to prove which is better than the other, since the UT in this PR is fairly simple.

Well, it does tell us that they are close in terms of inclusion overhead, which was my concern.

I think we should focus closely on the specific benefits that the test framework is expected to deliver for this project.

Agreed. Is there a detailed (hopefully unbiased) comparison somewhere?

[zhjwpku]

Thanks for the number @zhjwpku .

As the results shows, catch2 is better at preprocessed file size and compile time, but gtest is better when it comes to link time. I don't think the above tests is adequate to prove which is better than the other, since the UT in this PR is fairly simple.

Well, it does tell us that they are close in terms of inclusion overhead, which was my concern.

I think we should focus closely on the specific benefits that the test framework is expected to deliver for this project.

Agreed. Is there a detailed (hopefully unbiased) comparison somewhere?

I found this link: https://yurigeronimus.medium.com/guide-for-choosing-a-test-framework-for-your-c-project-2a7741b53317
Not sure if it is biased though.

[pitrou]

I found this link: https://yurigeronimus.medium.com/guide-for-choosing-a-test-framework-for-your-c-project-2a7741b53317

Seems like a lot of fluff with no substance, unfortunately.

[wgtmac]

LGTM. Thanks!

We can always improve it when we have better ideas.

[zhjwpku]

@raulcd @Fokko can you please review this PR?

[wgtmac]

BTW, should we run ctest in build_iceberg.sh (or a separate script) from pre-commit check in the CI?

[zhjwpku]

BTW, should we run ctest in build_iceberg.sh (or a separate script) from pre-commit check in the CI?

I've added the ctest command in build_iceberg.sh, this will run test on all platforms we are targeting.

[raulcd]

Thanks for the work @zhjwpku ! 👍

[wgtmac]

@Fokko @Xuanwo Should we merge this?

[wgtmac]

@zhjwpku It is better to change the PR title to reflect that we have chosen googletest?

[zhjwpku]

@zhjwpku It is better to change the PR title to reflect that we have chosen googletest?

Updated the PR title as well as the PR's first message, thanks for the suggestion ;)

[Xuanwo]

Thank you @zhjwpku for working on this, and thank you all for the review. Let's move.