Page MenuHome

Freestyle Python scripts update
ClosedPublic

Authored by Folkert de Vries (flokkievids) on Feb 15 2014, 9:15 PM.

Details

Summary

This patch is meant to update Freestyle's Python scripts to make full usage of the new features of Python and Freestyle's Python API.

Freestyle's Python scripts are pretty old already, and were never given much attention. With the 2.7x generation of Blender coming up, this is an excellent time to update Freestyle's Python scripts, hopefully adding some new features and achieving some speed improvements on the way.

main goals:

  • Use for loops where possible
  • general cleanup, making use of more recent python features (generators, ternary operator, ect.)
  • update the documentation on the way (it's lacking atm)

This builds upon earlier work by @Tamito Kajiyama (kjym3) and me.

Edit:
as of patch 4, this revision depends on D545

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
release/scripts/freestyle/modules/freestyle/predicates.py
366 ↗(On Diff #1502)

Same question as in line 360.

487 ↗(On Diff #1502)

Same comment here as in AndUP1D.

497 ↗(On Diff #1502)

Same comment here as in OrUP1D.

release/scripts/freestyle/modules/freestyle/shaders.py
90

Please, consider keeping the original order of classes to make code review easier.

161

Casting to float is not necessary in Python 3. An integer divided by another gives a float value.

215

Just a thought: This shader relies on vertex index to determine new stroke thickness. That would work fine if the vertex intervals are more and less the same; otherwise nonlinear thickness variation is expected. We might want to rely on the u parameter of the stroke instead, to assure smooth thickness changes.

232

Redundant casting to float: c should already be a float.

532

The constant is 0.01 in the original code.

636

How about using the notation stroke[0], stroke[-1] and so on here? That makes the creation of iterators unnecessary.

745

Using Iterface1D.vertices_begin() may further simplify the updated code (not tested):

it = stroke.vertices_begin()
for svert in it:
    svert.point += self._normal(it) * self._lambda * self._angle(it)

It is noted that the .object property of Interface0DIterator gives a StrokeVertex instance if the iterator is created from Stroke. Python does not have a dynamic_cast operator available in C++, so casting to StrokeVertex is automatically done when the .object property value is returned.

763

No return value is required.

793

Using stroke[0] and stroke[-1] would be easier.

815

An explicit int comparison qi == 0 is preferable to me.

976

A shorter form tuple(it) could be used here instead of (svert for svert in it), but maybe the former appears too cryptic. I prefer the latter.

1282

Is this strokeAsBezierShader class meant to be here?

1304

These 3 lines look like testing stuff. Please, remove them if they are unnecessary for a release.

release/scripts/freestyle/modules/freestyle/utils.py
78

Originally these iter_ helper functions were returning an iterator and extra object(s), because in the caller side, one might need to call 0D/1D functions which take an Interface0DIterator. It is true that returning a stroke vertex instead of an iterator saves one line of code (i.e., sv = it.object), but at the cost of creating an iterator in parallel to the iter_ loop on the caller side if it is necessary to call 0D/1D functions. To me returning an iterator instead of a stroke vertex seems a way to go since it will help make user code simple and straightforward although the extra line sv = it.object might appear clumsy.

152

The function name could be rgb_to_intensity or rgb_to_grayscale.

There are many named RGB-to-grayscale conversion methods (see http://en.wikipedia.org/wiki/Grayscale for more info), but the formula used here seems not any of them. I don't remember from where I took it...

204

This helper function seems not to work in the way the name suggests, since prev and svert are not being updated in the for loop.

release/scripts/freestyle/modules/parameter_editor.py
153 ↗(On Diff #1502)

How about a further simplification:

v1 = min(v1, fac * v2)
192 ↗(On Diff #1502)

The call of the .copy() method seems redundant:

point = -sv.point_3d.normalized()
257 ↗(On Diff #1502)

The whole updates of for iterations below can be revised after the iterator API revisions in D545, so I leave them for another round of review in the feature.

794 ↗(On Diff #1502)

This added condition seems wrong. If self._length_min is set to a number, then the length checking should be performed even if self._length_max is unset (and vice versa).

863 ↗(On Diff #1502)

Same comment as in line 794.

1082 ↗(On Diff #1502)

I prefer either if len(pattern) > 0 or if pattern.

thanks @Tamito Kajiyama (kjym3) for the code review. I've responded to a few of your comments and will upload a new revesion in a few moments.

release/scripts/freestyle/modules/freestyle/chainingiterators.py
236 ↗(On Diff #1502)

In what way? I didn't really test this but it seems to me that any() made for situations exactly like this. I would assume that it has also been optimized (memory usage ect.)

It shouldn't be slower and the code shows exactly what you want to achieve.

260 ↗(On Diff #1502)

tests with imagemagick over the past two months have not once given an error here. I will however look into it.

428 ↗(On Diff #1502)

We spoke about this earlier. there is a chance the iterator will turn back on itself, creating an infinite loop. This eliminates the chance of that happening.

release/scripts/freestyle/modules/freestyle/predicates.py
317 ↗(On Diff #1502)

it is not correct. probably forgot that while testing. using self._func(inter) will however result in an infinite loop printing "this viewmap does not exist", alike most, if not all, ...Viewmap...-methods.

because of this i've ommited it from my tests.

360 ↗(On Diff #1502)

Again, unable to test, executing this function enters an infinite loop printing "this steerable viewmap level does not exist"

release/scripts/freestyle/modules/freestyle/shaders.py
215

thought about this aswell. speedwise, it should barely matter (may be faster even, albeit mininally)

It would however make regression testing inaccurate. I propose to stick to the old approach for now, verify that all works well. Then we can implement the use of the u parameter.

763

in python, explicit is said to be better than implicit.
I like this better than the empty 'return'. it is explicit about what it returns.

976

actually, verticesToRemove is a generator object here. altough the effect in this case may be minimal, I think it is a good practice to use generators when possible.

1282

No. this was an experiment with your similar shader. we can look into inclusion of such a shader after this patch gets applied.

Folkert de Vries (flokkievids) updated this revision to Unknown Object (????).Jun 10 2014, 11:16 PM

update after review by @Tamito Kajiyama (kjym3). This update doesn't include parameter_editor.py. I think it will be more effective to update that once the api has been finished.

this revision now relies on D545

Folkert de Vries (flokkievids) updated this revision to Unknown Object (????).Jun 10 2014, 11:18 PM

based on review by @Tamito Kajiyama (kjym3). This diff doesn't contain parameter_editor.py, because I think it will be more effective to update that once the API has been finished/more settled.

This revision now relies on D545

Hi @Folkert de Vries (flokkievids), please find below a few notes in reply to some of your comments.

release/scripts/freestyle/modules/freestyle/chainingiterators.py
236 ↗(On Diff #1502)

You are right, it looks like I was confused with all().

428 ↗(On Diff #1502)

Thanks for the reminder. The code revision looks okay.

release/scripts/freestyle/modules/freestyle/shaders.py
763

I agree that explicit is better than implicit in Python. In this case, however, the shade() method is not supposed to return a value. Explicitly returning None appears weird to me.

976

I agree with you that using generators when possible is a good practice.

Another concern I got is whether using a generator here is safe or not. The generator (svert for svert in it) keeps a pointer to a StrokeVertex in a sequence of objects, and does not keep all references of the StrokeVertex objects to be removed. This situation is similar to a common coding error in Python where items are removed from a list within a loop over the list item:

for x in somelist:
    somelist.remove(x)

Since Python programmers are familiar with this common mistake, it would appear straightforward that verticesToRemove is a genuine list or tuple.

Hi @Folkert de Vries (flokkievids), thanks for the patch updates. Below you will find a few additional remarks on the new code revisions.

release/scripts/freestyle/modules/freestyle/predicates.py
246 ↗(On Diff #1881)

I liked the expression sum(1 for ve in it if func(it) > self._a) in a previous revision. Is there any reason the new form is preferable?

release/scripts/freestyle/modules/freestyle/shaders.py
683

These nested for loops are difficult to read... Why not iterate self._nbIter-1 times and creating a new Interface0DIterator instance inside the loop?

for i in range(1, self._nbIter):
    it = Interface0DIterator(stroke)
    for svert in it: # this works in Python
        svert.point += self._normal(it) * self._lambda * self._angle(it)
979

This assignment seems redundant.

1011

Please, remove this debug print.

Tamito Kajiyama (kjym3) requested changes to this revision.Jun 12 2014, 3:34 AM
Folkert de Vries (flokkievids) updated this revision to Unknown Object (????).Jun 14 2014, 11:13 PM

more fixes for problems pointed out by @Tamito Kajiyama (kjym3)

I've also done some performance checks, getting very nice results overall.
especially for the pyBluePrint* shaders the speedup is about really noticable:

pyBluePrintCirclesShader 105.1026% 0.20101094245910645
pyBluePrintEllipsesShader 32.52572% 0.38302183151245117
pyBluePrintSquaresShader 66.66955% 0.11000704765319824
pyBluePrintDirectedSquaresShader 32.35314% 0.1350080966949463
rendered: 41 44 in 0.8290479183197021 0.5550310611724854
delta 0.274017 s, 49.369644% speed increase, total 1.384079 s

The ammount of function calls is also substantially lower (mainly because it.increment() is now handled in C, but still.)

Below you will find a few additional review comments. I feel like the patch is almost ready for inclusion into Git master.

release/scripts/freestyle/modules/freestyle/chainingiterators.py
100 ↗(On Diff #1913)

Just a thought: We could define a local function (predicate) and use filter built-in function to make this generator-based expression:

mate_id = vertex.get_mate(self.current_edge).id

def is_mate(ve):
    return ve.id == mate_id

return next(filter(is_mate, it), None)

The predicate can be an unnamed lambda function in this case, there are similar cases in other chaining iterators that could employ the same idiom. Regular use of the same idiom is preferable.

184 ↗(On Diff #1913)

Minor comment: not result and bpy.app.debug_freestyle would a bit faster.

333 ↗(On Diff #1913)

Should be TVertex. Please, double-check all chaining iterators for this typo.

release/scripts/freestyle/modules/freestyle/predicates.py
482 ↗(On Diff #1913)

Should be if not self._predicates:.

492 ↗(On Diff #1913)

Same as the comment in line 482.

538 ↗(On Diff #1913)

This comment is not always true, since pyNatureBP1D is mostly meant to be used as an argument of an API function that takes a 1D binary predicate.

release/scripts/freestyle/modules/freestyle/shaders.py
171

c = i / n

195

c = i / n

358

Now this can be it = Interface0DIterator(stroke).

422

c = 1.0 - 2.0 * abs(i / n - 0.5)

460
it = Interface0DIterator(stroke)
for svert in it:
  mat = self._func(it)
529
it = Interface0DIterator(stroke)
for svert in it:
  c = func(it)
683

Please, consider revising these nested for loops as suggested in https://developer.blender.org/D319?id=1881#inline-1886.

749

it2 = it.incremented()

771
it = Interface0DIterator(stroke)
for svert in it:
    normal = self._getNormal(it)
864

I believe verticesToRemove = tuple(svert for svert in it) is safer (see https://developer.blender.org/D319?id=1502#inline-1884 ).

Tamito Kajiyama (kjym3) requested changes to this revision.Jun 22 2014, 6:08 AM
Folkert de Vries (flokkievids) updated this revision to Unknown Object (????).Jun 22 2014, 6:33 PM

More fixes after code review. I was under the impression that I had already uploaded some of those, but that turned out not to be the case.

furthermore, I've realised a speedup of over 100% for pyBluePrintCirclesShader and pyBluePrintEllipsesShader by using an lru-cache.
previously, cos and sin were called tens of thousands of times for relatively simple scenes. This turned out to be unnessecary. using the lru-cache the ammount of calls is decreased even further, resulting in the massive (and quite scalable) speedup.

finally, the shaders for caps have now been included in shaders.py and optimized.

release/scripts/freestyle/modules/freestyle/chainingiterators.py
100 ↗(On Diff #1913)

I prefer using a generator expression over an anonymous function and a barely-used built-in.
I'll look into a broader usage of the idiom.

184 ↗(On Diff #1913)

Just interested here: would it really, significantly matter?
Adjusted in rev. 7

release/scripts/freestyle/modules/freestyle/shaders.py
864

I hadn't noticed yet. a change of test scene revealed this problem. fixed it now

release/scripts/freestyle/modules/freestyle/chainingiterators.py
184 ↗(On Diff #1913)

No, not really. My comment was mostly for consistency (in other cases, we have put bpy.app.debug_freestyle at the end of conditions).

I find the patch quite ready for inclusion into Git master, with a few very minor comments. Feel free to update the revision to correct the typos and so on. Otherwise I will correct them when I commit the changes.

release/scripts/freestyle/modules/freestyle/predicates.py
482 ↗(On Diff #1982)

"Expected two or more BinaryPredicate1D"

492 ↗(On Diff #1982)

"Expected two or more BinaryPredicate1D"

release/scripts/freestyle/modules/freestyle/shaders.py
1004

I guess p_mean = (1 / n) * sum(svert.point for svert in stroke) would suffice?

release/scripts/freestyle/modules/freestyle/utils.py
47

I guess this helper function needs to be renamed to chromatize as referred to in the code below?

Thanks for such a quick code review. I'll update the typos in a moment.

release/scripts/freestyle/modules/freestyle/shaders.py
1004

It doesn't, sadly. it raises:
AttributeError: Vector addition: (int + Vector) invalid type for this operation
including the empty Vector as a starting point makes this work.

release/scripts/freestyle/modules/freestyle/utils.py
47

I'm still unsure about the name. do you have a prefference?

release/scripts/freestyle/modules/freestyle/shaders.py
1004

Ok, sum() takes two arguments and the second is the initial value, which in this case should be Vector((0, 0)). No problem then.

release/scripts/freestyle/modules/freestyle/utils.py
47

I don't really have much preference: rgb_to_grayscale, rgb_to_intensity, chromatize, or whatever makes sense. One option is rgb_to_bw() since the same formula appears in source/blender/blenlib/intern/math_color_inline.c.

56

I saw this helper function also in your regression test suite. Do you actually want to include this in the freestyle.utils module?

All right, that's mostly it.

One final problem is the leaking of imports into the freestyle submodules.
for instance, type 'import freestyle;dir(freestyle.shaders)' into the console and you'll see that all imports made in shaders.py
leak through. Are there feasible ways to prevent this?

release/scripts/freestyle/modules/freestyle/utils.py
47

took that one

56

I think it's not needed. (pretty nice recipe though)

I would not consider it's a leakage of names. What we see from dir(freestyle.shaders) is actually the list of names available in the intra-module global name space of the freestyle.shaders module. We can reduce the number of names to the minimum, but trying to make it as small as possible is likely unpractical. For instance, we can import names from other submodules within the definition of a method, at the cost of repeated imports of the same names here and there within shaders.py. Maybe that is not what we want.

Folkert de Vries (flokkievids) updated this revision to Unknown Object (????).Jun 23 2014, 2:07 PM

More fixes, notably:

  • cleaning up utils.py
  • using a seperate function in chaining iterators for finding a matching vertex

regarding the imports in the freestyle module (and the 'leaking' of their names): the bpy module (scripts/modules/bpy/utils.py for instance)
imports with an underscore, so when the bpy.utils module is imported, those names don't import with it.

This works, but it isn't very readable/convenient, so I guess we'll just leave it as is for freestyle.

Everything looks fine. If you agree, I will commit the latest revision into Git master.

I see no further problems, so please do commit.