This seems to be a nicer improvement already!

Is it possible to group based on source file, to make behavior closer to as if there was not uber-binary approach? The reason for this is because foo_test.cc might define different fixture names which are testing effectively different aspects of the same underlying codepaths (as in, they make up for code coverage, but might be different fixtures for various reasons).

Actually. Maybe it's possible to make blender_add_test_executable to not "surgeon" on the tests suits at all, and for blender_add_test_lib use the logic you did here?

The GTest discovery works by actually running blender_tests --gtest_list_tests and parsing the output. This is of course only possible after it's been built, which means that we have no info any more as to which module the tests belonged to.

I can modify gtest_add_tests, which tries to run some regexps on the source code in order to find tests. This is a bigger change, because at the time of calling gtest_add_tests there must already be a CMake target defined for the test executable. This means that we have to reorder the root CMakeLists.txt to define the test runner first, and then the rest of Blender. I did this locally as a quick test, and it seems to work well.

I keep forgetting to mention.
On macOS there is some very measurable overhead when ctest is running a lot of binaries. Not sure details of why is it so, but less invocations ctest does is better here.

As for the issue Ray mentioned, is almost as if it's more efficient if Ray gives a hint of whet to modify or even provides a patch? Unless there is an easy way to check changes on Linux.

After this patch, the test code needs to be cleaned up & clarified, as it's turning into a nice moist ball of spaghetti: T83222: Clean up CMake scripts for automated tests.

On Linux - Pop!_OS 20.04 LTS - AMD Ryzen 9 - 24 logical cores

Before patch

step            elapsed
----            -------
update-code     00:00:03.3398348
compile-code    00:03:50.3780962
test-code-unit  00:02:49.4274320
test-code-smoke 00:00:05.8998064

After patch

step            elapsed
----            -------
clean-code      00:00:00.0112528
compile-code    00:02:44.5886884
test-code-unit  00:01:59.2731646
test-code-smoke 00:00:05.8164091

Although I like the test-code-unit numbers, there is also a similar difference in compile-code time. This may indicate that there was something else going on in the system, as my patch shouldn't cause that difference in compile time.

at 2-3 minutes each perhaps those were an incremental builds?

Yes, incremental build with partial clean (test artifacts only).

The code builds times are only relevant with full clean builds.
However, they are good to track incrementally also, especially if hitting the same builder for each commit.

As a note, the build bot scripts only deletes the install folder and not the build and the test artifacts on the machines.
I do this in the new pipeline in my first step before calling the build bot python scripts.

I normally like to ensure forced full clean builds and then skip it when needed.

On Windows - 10 Pro 1909 - Intel i7 @ 2.3Ghz - 16 logical cores

Before Patch

step             elapsed
----             -------
clean-code       00:00:00.0428429
update-code      00:01:56.6479339
compile-code     00:43:58.8301665
test-code-unit   00:06:30.4856439
test-code-smoke  00:00:09.4788121
package-binaries 00:01:06.3493870

After Patch

step           elapsed
----           -------
clean-code     00:00:00.0004629
update-code    00:00:05.2854835
patch-code     00:00:09.4928776
compile-code   00:05:26.2016864
test-code-unit 00:06:12.8964456

After patch, 1 test failed, did not continue the pipeline
Will run the tests again, without patch from master.
But pretty sure it's svn not on the correct commit vs git commit.
High compile time on windows is casued by CUDA building.

@Sebastián Barschkis (sebbas) could you give this a quick test on macOS to see if it breaks anything?

Yes, I will check that!

Ok, so with the patch applied the modifiers and physics_cloth tests are failing for me on macOS. Will investigate.

it's a pretty old patch, you may just need to merge master if you applied the patch with arc.