So the tests would ship with blender? or are we going to build 2 blenders on the bot, one for testing and one for distro?

@Ray Molenkamp (LazyDodo) Good question. I don't see much harm in shipping with the tests enabled, although many of them won't run without the supporting files in SVN.

anyone aware of other software products that ship unit tests in their release binaries?

For the namespace question: I think it should be tests sub-namespace of what's being tested. For example, blender::depsgraph::tests.
When some helpers are needed use anonymous namespace inside of the tests.

For the rest, seems quite clear implementation actually. Only lacking some explanation which will help adding new tests.

I had to add this to make it build, but I guess this is still work in progress (and not integrated with ctest?).

diff --git a/tests/gtests/CMakeLists.txt b/tests/gtests/CMakeLists.txt
index 68a44b28817..90f610899b1 100644
--- a/tests/gtests/CMakeLists.txt
+++ b/tests/gtests/CMakeLists.txt
@@ -21,9 +21,13 @@ if(WITH_GTESTS)
   endif()
   unset(_test_libs)

+  setup_libdirs()
+
   # This builds `bin/tests/blender_test`.
   BLENDER_SRC_GTEST(blender "blender_test.cc" "${TEST_LIBS}")

+  setup_liblinks(blender_test)
+
   # Build the not-yet-ported tests
   include(GTestTesting)

Other than that looks good so far.

I had to disable WITH_LIBMV to even get it through cmake

This patch does not pass the CMake step for me either, afaict it's missing the BLENDER_SRC_GTEST entirely?

In D7649#192674, @Sybren A. Stüvel (sybren) wrote:

I think it's nice to have the tests in the same namespace as the code under test. It could be in the global namespace too and have a using blender::module; statement at the top. I don't have strong feelings about this one.

Personally that's what I would do, put it in the global namespace just like the C code it is testing. When testing C++ code in some namespace the tests can also be in the corresponding namespace. I don't feel strongly about this either though.

It's integrated with ctest on my machine. Running ctest -R blender -V shows that the mat3_vec_to_roll and evaluate_fcurve tests are run from blender_test. Is there any other way of running CTest that I should test with?

I was expecting the tests to remain individual ctests instead of all being part of one Blender ctest, invoking blender_tests multiple times with different --gtest_filter arguments.

In D7649#192724, @Brecht Van Lommel (brecht) wrote:

I was expecting the tests to remain individual ctests instead of all being part of one Blender ctest, invoking blender_tests multiple times with different --gtest_filter arguments.

Ah, that clarifies things. Of course this can be done, but it has some downsides:

• Each test invocation then has to load the same binary. I expect this to be potentially quite slow, as it can be 1.5 GB in debug mode. Loading that binary once and then running all tests will probably be faster, although it does increase the chance of one test influencing the test (as they'd then all run in the same process) and would prevent running the tests in parallel.
• So far, the CMake commands that construct the tests only handle source & library filenames. The names of the tests are unrelated to those, so in order to have CMake generate individual add_test() calls, test names would either have to be repeated in the CMakeLists.txt files, or we'd need some way to obtain them from the input source or the generated test library.

Since the Python tests also start Blender many times over, I don't think the first point is that big. The second, however, is more of an issue.

In D7649#192868, @Sybren A. Stüvel (sybren) wrote:

• Each test invocation then has to load the same binary. I expect this to be potentially quite slow, as it can be 1.5 GB in debug mode. Loading that binary once and then running all tests will probably be faster, although it does increase the chance of one test influencing the test (as they'd then all run in the same process) and would prevent running the tests in parallel.

I'm pretty sure the OS will mmap and load just the required symbols, so it's shouldn't be that bad.

• So far, the CMake commands that construct the tests only handle source & library filenames. The names of the tests are unrelated to those, so in order to have CMake generate individual add_test() calls, test names would either have to be repeated in the CMakeLists.txt files, or we'd need some way to obtain them from the input source or the generated test library.

Maybe we can use gtest_discover_tests? Or copy the code for that from GoogleTest.cmake and tweak it if we need it to work a little different.
https://cmake.org/cmake/help/git-stage/module/GoogleTest.html

That or use a naming convention so every test in a file starts with the same prefix, but that's a bit fragile.

That or use a naming convention so every test in a file starts with the same prefix, but that's a bit fragile.

I would strongly suggest against using name.
You want to test any complex module with all sort of tests, and it would be wrong to stick to a single test case name, or to a single test fixture (and you can not name test case and test fixture the same name). Having an arbitrary prefix for all tests seems annoying super-imposed requirement which can be avoided.