Page MenuHome
Differential D5357 Diff 16957 tests/python/modules/mesh_test.py

Changeset View

Standalone View

View Options

tests/python/modules/mesh_test.py

  • This file was added.
# ##### BEGIN GPL LICENSE BLOCK #####
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software Foundation,
# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# ##### END GPL LICENSE BLOCK #####
# <pep8 compliant>
# A framework to run regression tests on mesh modifiers and operators based on howardt's mesh_ops_test.py
#
# General idea:
# A test is:
# Object mode
# Select <test_object>
# Duplicate the object
# Select the object
# Apply operation for each operation in <operations_stack> with given parameters
# (an operation is either a modifier or an operator)
zazizizouAuthorUnsubmitted
Done
Inline Actions

I use the term 'operation' as a generalization for operator and modifiers. Is there a better word for that?

zazizizou: I use the term 'operation' as a generalization for operator and modifiers. Is there a better…
brechtUnsubmitted
Done
Inline Actions

Operation sounds good to me.

brecht: Operation sounds good to me.
# test_mesh = <test_object>.data
# run test_mesh.unit_test_compare(<expected object>.data)
# delete the duplicate object
#
# The words in angle brackets are parameters of the test, and are specified in
# the main class MeshTest.
#
# If the environment variable BLENDER_TEST_UPDATE is set to 1, the <expected_object>
# is updated with the new test result.
# Tests are verbose when the environment variable BLENDER_VERBOSE is set.
import bpy
import os
class ModifierSpec:
"""
Holds one modifier and its parameters.
"""
def __init__(self, modifier_name: str, modifier_type: str, modifier_parameters: dict, apply_modifier: bool):
"""
Constructs a modifier spec.
:param modifier_name: str - name of object modifier, e.g. "myFirstSubsurfModif"
:param modifier_type: str - type of object modifier, e.g. "SUBSURF"
:param modifier_parameters: dict - {name : val} dictionary giving modifier parameters, e.g. {"quality" : 4}
:param apply_modifier: bool - True if we want to apply the modifier right after adding it to the object.
"""
self.modifier_name = modifier_name
self.modifier_type = modifier_type
self.modifier_parameters = modifier_parameters
self.apply_modifier = apply_modifier
def __str__(self):
return "Modifier: " + self.modifier_name + " of type " + self.modifier_type + \
" with parameters: " + str( self.modifier_parameters )
class OperatorSpec:
"""
Holds one operator and its parameters.
"""
def __init__(self, operator_name : str, operator_parameters : dict):
"""
Constructs an operatorSpec
:param operator_name: str - name of mesh operator from bpy.ops.mesh, e.g. "bevel" or "fill"
:param operator_parameters: dict - {name : val} dictionary containing operator parameters.
"""
self.operator_name = operator_name
self.operator_parameters = operator_parameters
def __str__(self):
return "Operator: " + self.operator_name + " with parameters: " + str(self.operator_parameters)
class MeshTest:
"""
A mesh testing class targeted at testing modifiers and operators. It holds a stack of mesh operations, i.e.
modifiers or operators. The test is executed using the public method run_test.
"""
def __init__(self, test_object_name : str, expected_object_name : str, operations_stack=None):
"""
Constructs a MeshTest object. Raises a KeyError if objects with names expected_object_name
or test_object_name don't exist.
:param test_object: str - Name of object of mesh type to run the operations on.
:param expected_object: str - Name of object of mesh type that has the expected
geometry after running the operations.
:param operations_stack: list - stack holding operations to perform on the test_object.
"""
if operations_stack is None:
operations_stack = []
self.operations_stack = operations_stack
self.verbose = os.environ.get("BLENDER_VERBOSE") is not None
self.update = os.getenv('BLENDER_TEST_UPDATE') is not None
# initialize test objects
objects = bpy.data.objects
self.test_object = objects[test_object_name]
self.expected_object = objects[expected_object_name]
if self.verbose:
print("Found test object {}".format(test_object_name))
print("Found test object {}".format(expected_object_name))
# private flag to indicate whether the blend file was updated after the test
self._test_updated = False
def set_test_object(self, test_object_name):
"""
Set test object for the test. Raises a KeyError if object with given name does not exist.
:param test_object_name: name of test object to run operations on.
"""
objects = bpy.data.objects
brechtUnsubmitted
Done
Inline Actions

Style: begin comment with capital, end with dot.

brecht: Style: begin comment with capital, end with dot.
self.test_object = objects[test_object_name]
def set_expected_object(self, expected_object_name):
"""
Set expected object for the test. Raises a KeyError if object with given name does not exist
:param expected_object_name: Name of expected object.
"""
objects = bpy.data.objects
self.expected_object = objects[expected_object_name]
def add_modifier(self, modifier_spec : ModifierSpec):
"""
Add a modifier to the operations stack.
:param modifier_spec: modifier to add to the operations stack
"""
self.operations_stack.append( modifier_spec )
if self.verbose:
print("Added modififier {}".format(modifier_spec))
def add_operator(self, operator_spec : OperatorSpec):
"""
Adds an operator to the operations stack.
:param operator_spec: OperatorSpec - operator to add to the operations stack.
"""
self.operations_stack.append( operator_spec )
def _on_failed_test(self, compare, evaluated_test_object):
if self.update:
if self.verbose:
print("Test failed expectantly. Updating expected mesh...")
# replace expected object with object we ran modifiers on, i.e. evaluated_test_object.
evaluated_test_object.location = self.expected_object.location
expected_object_name = self.expected_object.name
bpy.ops.object.select_all(action="DESELECT")
self.expected_object.select_set(True)
bpy.ops.object.delete()
evaluated_test_object.name = expected_object_name
# save file
blender_file = bpy.data.filepath
bpy.ops.wm.save_as_mainfile(filepath=blender_file)
self._test_updated = True
self.expected_object = evaluated_test_object
return True
else:
if self.verbose:
blender_file = bpy.data.filepath
blender_executable = bpy.app.binary_path
bpy.ops.wm.save_as_mainfile(filepath=blender_file)
print("Test failed with error: {}. Resulting object mesh {} did not match expected object {} "
"from file blender file {}".
format(compare, evaluated_test_object.name, self.expected_object.name, blender_file))
zazizizouAuthorUnsubmitted
Done
Inline Actions

Ideally, I would do something like:

  • select relevant objects.
  • hide all non-selected objects.

This way, blender opens with only failing meshes visible. But I'm not sure how to write a loop to pass it in the command line. Any suggestion on how to do this better? Or is this good enough?

zazizizou: Ideally, I would do something like: - select relevant objects. - hide all non-selected objects.
brechtUnsubmitted
Done
Inline Actions

You could put this code in a separate .py file and then pass arguments like:

blender -P file.py -- a b c

And then read them from sys.argv.

brecht: You could put this code in a separate .py file and then pass arguments like: blender -P file.
print("Run following command to open blender with failing objects selected:")
brechtUnsubmitted
Done
Inline Actions

This can be done without operators:

bpy.data.objects.remove(self.expected_object, do_unlink=True)
brecht: This can be done without operators: ``` bpy.data.objects.remove(self.expected_object…
print()
# Print command to hide all objects except failing ones.
python_expr = "\"import bpy; " \
"bpy.ops.object.select_all(action='SELECT'); " \
"test_object = bpy.data.objects[\'{}\']; " \
"expected_object = bpy.data.objects[\'{}\']; " \
"evaluated_test_object = bpy.data.objects[\'{}\']; " \
"test_object.select_set(False); " \
"expected_object.select_set(False); " \
"evaluated_test_object.select_set(False);" \
"[sel.hide_set(True) for sel in bpy.context.selected_objects]\"".\
format(self.test_object.name, self.expected_object.name, evaluated_test_object.name)
brechtUnsubmitted
Done
Inline Actions

The test.blend should not be saved when there is no update. Version control should not show the file as modified.

The way I imagined this to work is that it gives you a command to actually run the test. This is needed also to easily reproduce crashes, when it can't save a .blend.

Perhaps the simplest would be to add an additional parameter to the original command line, to run just one test, like:

blender modifiers.blend --python modifiers.py -- --show-test 25

Which could be constructed like:

command = list(sys.argv)
command.remove("--background")
command.append("--")
command.append("--show-test")
command.append("25")

Then bevel_operator.py, boolean_operator and modifiers.py should be simplified to just constructing a list of tests then, and leaving logic including like sys.argv parsing to mesh_test.py. But I think that's a good thing anyway.

It also means you can avoid passing that long Python expressions.

brecht: The test.blend should not be saved when there is no update. Version control should not show the…
zazizizouAuthorUnsubmitted
Done
Inline Actions

I think generalising this in the MeshTest class would make it too complex. The general case has to consider

  • N test objects, M operators and P modifiers

But tests usually either test

  • 1 object and N operations or
  • N objects and 1 operation

So I would prefer to keep the failed tests handling on the modifiers.py side and not in the MeshTest class if that's ok with you. To deduplicate code, I introduced the helper classes OperatorTest to handle the typical case of N objects and 1 operator.

zazizizou: I think generalising this in the `MeshTest` class would make it too complex. The general case…
brechtUnsubmitted
Done
Inline Actions

I'm fine with it. We can always make it more generic later if needed.

brecht: I'm fine with it. We can always make it more generic later if needed.
print(blender_executable + " " + blender_file + " --python-expr " + python_expr)
return False
def is_test_updated(self):
"""
Check whether running the test with BLENDER_TEST_UPDATE actually modified the .blend test file.
:return: Bool - True if blend file has been updated. False otherwise.
"""
return self._test_updated
def _apply_modifier(self, test_object, modifier_spec : ModifierSpec):
"""
Add modifier to object and apply (if modifier_spec.apply_modifier is True)
:param test_object: bpy.types.Object - Blender object to apply modifier on.
:param modifier_spec: ModifierSpec - ModifierSpec object with parameters
"""
modifier = test_object.modifiers.new(modifier_spec.modifier_name,
modifier_spec.modifier_type)
if self.verbose:
print("Created modifier {} of type {}.".
format(modifier_spec.modifier_name, modifier_spec.modifier_type))
for param_name in modifier_spec.modifier_parameters:
try:
setattr(modifier, param_name, modifier_spec.modifier_parameters[param_name])
if self.verbose:
print("set parameter '{}' with value {}".
format(param_name, modifier_spec.modifier_parameters[param_name]))
except AttributeError:
if self.verbose:
print("No modifier parameter {}".format(param_name))
# clean up
bpy.ops.object.delete()
if modifier_spec.apply_modifier:
bpy.ops.object.modifier_apply(modifier=modifier_spec.modifier_name)
def run_test(self):
"""
Apply operations in self.operations_stack on self.test_object and compare the
resulting mesh with self.expected_object.data
:return: bool - True if the test passed, False otherwise.
"""
self._test_updated = False
# Duplicate test object
bpy.ops.object.mode_set(mode="OBJECT")
bpy.ops.object.select_all(action="DESELECT")
bpy.context.view_layer.objects.active = self.test_object
self.test_object.select_set(True)
bpy.ops.object.duplicate()
evaluated_test_object = bpy.context.active_object
evaluated_test_object.name = "evaluated_object"
if self.verbose:
print(evaluated_test_object.name, "is set to active")
# Add modifiers and operators
for operation in self.operations_stack:
if isinstance(operation, ModifierSpec):
self._apply_modifier(evaluated_test_object, operation)
elif isinstance(operation, OperatorSpec):
# TODO: Operator test
pass
else:
raise ValueError("Expected operation of type {} or {}. Got {}".
format(type(ModifierSpec), type(OperatorSpec),
type(operation)))
# compare resulting mesh with expected one
test_mesh = evaluated_test_object.data
expected_mesh = self.expected_object.data
compare = test_mesh.unit_test_compare(mesh=expected_mesh)
success = (compare == 'Same')
zazizizouAuthorUnsubmitted
Done
Inline Actions

When tests are expected to fail, I still return False whereas some other tests return True (e.g. bevel). I think it's better to update the expected object and return False. This way it's more clear that the test was updated. When BLENDER_TEST_UPDATE is set to 1 and nothing gets updated, i.e. tests pass, then we return True.

zazizizou: When tests are expected to fail, I still return False whereas some other tests return True (e.g.
brechtUnsubmitted
Done
Inline Actions

To find out which tests were updated, "svn status" is already there. I find it more convenient if it runs the test again after updating and return the result of that.

Otherwise you have to run tests 3 times, once to discover the failure, once to update and once to verify it all passes.

brecht: To find out which tests were updated, "svn status" is already there. I find it more convenient…
zazizizouAuthorUnsubmitted
Done
Inline Actions

Running the tests 3 times does indeed sound bad. I updated the MeshTest class but it is up to the caller of MeshTest to check if the test has been updated and act accordingly (see modifiers.py)

zazizizou: Running the tests 3 times does indeed sound bad. I updated the MeshTest class but it is up to…
if success:
if self.verbose:
print("Success!")
# cleanup
if self.verbose:
print("Cleaning up...")
bpy.ops.object.delete() # delete evaluated_test_object
return True
else:
return self._on_failed_test(compare, evaluated_test_object)
brechtUnsubmitted
Not Done
Inline Actions

Always print this information, even if BLENDER_VERBOSE is not set.

brecht: Always print this information, even if `BLENDER_VERBOSE` is not set.
brechtUnsubmitted
Not Done
Inline Actions

Same comment as above.

brecht: Same comment as above.