Some quick pass.

Is there an easy way to update this test after some change which has expected behavior changes was made?

Working on "simulated user input"...

Bad API

I'm replacing @Sergey Sharybin (sergey) by his request. There are a few things that I find a bit strange.

• open the transform_regression.blend file in ../lib/tests/modeling/
• run the script on the file

This is the script in question:

import sys
import os

blender_src_dir = os.path.join(__file__[:__file__.find("lib")], "blender")
tests_python_dir = os.path.join(blender_src_dir, "tests", "python")
sys.path.append(tests_python_dir)

def set_args(run_all_tests=True, run_test=0):
    if not "--keep-open" in sys.argv:
        sys.argv.append("--keep-open")

    if run_all_tests:
        if "--run-tests" in sys.argv:
            i = sys.argv.index("--run-tests")
            sys.argv.pop(i)
            sys.argv.pop(i)

        if not "--run-all-tests" in sys.argv:
            sys.argv.append("--run-all-tests")
    else:
        if "--run-all-tests" in sys.argv:
            sys.argv.remove("--run-all-tests")
        if not "--run-test" in sys.argv:
            sys.argv.append("--run-test")
            sys.argv.append("0")

    if "--run-test" in sys.argv:
        i = sys.argv.index("--run-test")
        sys.argv[i + 1] = str(run_test)

def main():
    import imp
    import transform_operators

    set_args(run_all_tests=True, run_test=9)

    imp.reload(transform_operators)
    print('-----------------')
    os.environ["BLENDER_VERBOSE"] = "1"
    transform_operators.main()

if __name__ == "__main__":
    main()

• The code has no comments that explains what happens, or what the file is even for.
• There is no comment about the import transform_operators, and which module/file it expects to be importing.
• The expression os.path.join(__file__[:__file__.find("lib")], "blender") is hard to read and very fragile (it breaks for a user with files in /home/libby/). I would suggest using pathlib instead of os.path.
• The imp module has been deprecated since Python 3.4 and should definitely not be used in new code.
• There is just one single call to set_args(), so I don't see the need to even have defaults for the parameters. Use type declarations instead, if you want to clarify their type (👍 from me there).
• There can apparently be conflicting CLI arguments in sys.argv, which are silently replaced. This is always a bad situation, especially in a testing framework. Conflicting arguments should cause an error situation. If this error situation cannot be fixed at the root, at least there should be an explanation what exactly is the problem, why it cannot be resolved, and why this is silently corrected in this particular spot.

The transform_test.py has no documentation either. A class like TestCompare should really have some explaination about what it represents, and how it's intended to be used. There is also no need to use __slots__; they're nice to reduce memory overhead, but that's only relevant when lots of instances will be created. On the other hand, they do add redundant code (repetition of each attribute) that needs to be reviewed & maintained.

In transform_test.py there are no type declarations. This makes it harder than necessary to understand what is happening. For example, self.on_error = None gives no information beyond that the attribute exists.

Why is transform_operators.py not using Python's unittest module?

When it comes to writing clean code, there are also a bunch of improvements to make, but I think it's more important to get the fundamentals right first.

To drive the point home about a lack of documentation: right now it's not even clear whether transform_test.py is about transforming tests from one form into another, or whether they're testing the transform system.

In D7489#266994, @Germano Cavalcante (mano-wii) wrote:

I used another file as a reference (it also doesn't use the unittest module).
I confess that I don't remember much about how to use this module, so it will take me a while to implement it.

This is my personal opinion, but I think we should be using built-in functionality of Python unless there is a good reason to deviate.

The script in the file is not supposed to be used by Blender.
It is a utility for the developer to use.

That's only relevant when it's actually documented as such.

So the directory pointed to in tests_python_dir is not really that problematic.

Again, that's only the case if it's easy to read and understand.

The _TestCompare.run function has quite a high cognitive complexity. I think it's possible to greatly simplify the function by spliting some code off into smaller functions. That would leave a run() function like this:

def run(self, index, verbose=True):
    if verbose:
        print()
        print("Test comparison {}: {}".format(index, self.description))

    compare_success = True
    compare_result = ""

    if self.compare_matrix:
        compare_success, compare_result = self._compare_matrix()
    if compare_success and self.compare_data:
        compare_success, compare_result = self._compare_data()

    if not compare_success:
        print("Test comparison result: {}".format(compare_result))
        print("Resulting object '{}' did not match expected object '{}' from file {}".
              format(self.obj_a.name, self.obj_b.name, bpy.data.filepath))
        return False

    if verbose:
        print("Success!")
    return True

Something similar can be done for _dispatch_test_events_func(), as that's also a much too complex function.

In D7489#267004, @Howard Trickey (howardt) wrote:

I will second the idea that it should be OK to put a script in the .blend file to aid the developer.

I agree, but only if it's documented as such.