Page MenuHome

WM notifier queue optimization
AbandonedPublic

Authored by Erik Abrahamsson (erik85) on Dec 18 2020, 10:36 PM.

Details

Summary

The event duplicate check can be quite slow when there are lots of notifiers, for example when adding a huge amount of objects from a script.
By reversing the loop duplicates will be found faster.

Profiling:

Diff Detail

Repository
rB Blender

Event Timeline

Erik Abrahamsson (erik85) requested review of this revision.Dec 18 2020, 10:36 PM
Erik Abrahamsson (erik85) created this revision.
Erik Abrahamsson (erik85) edited the summary of this revision. (Show Details)
Brecht Van Lommel (brecht) retitled this revision from WM Event queue optimization to WM notifier queue optimization.Dec 21 2020, 1:22 PM
Brecht Van Lommel (brecht) edited the summary of this revision. (Show Details)

This patch is on the bare side, i may work just fine, but the description is on the lacking side, beyond "look it's faster now, here's some out of context screenshots" is virtually gives us nothing to work with. Yes there's two screenshots, but for all i know they are from different work loads.

I'd like to see at-least the following for performance related patches

  • A test script/scene that was used for profiling so reviewers can independently verify your results
  • At least mention the speedup in the text of the patch, a 5->1.5s speedup is impressive, it's ok mention it, making me go look for that data in barely readable screenshots is "not great", also not everyone may be familiar with the profiler your used, so extracting that information would be helpful, the screenshot could have been replaced with

running attached scripts:

beforeafter
script1.py5.0s1.5s (-3.5s)
script2.py4.2s4.4s (+02s
  • Do the tests still pass?
  • Are there any side effects? ie a significant change in the runtime either good or bad of the unit tests would be relevant to know

For future patches it's good to keep the guidance in the ingredients for a patch in mind.

The wm_event_do_notifiers part of that profile should be eliminated by rB4b46afae0951: WM: minor optimization for when there is a large number of notifiers.

For the rest, I think it's possible to write a Python script that is significantly worse in either case. If you send notifiers AAABBBCCC then backward will be faster. If you send ABACADAE then backward will be slower.

Fundamentally it's the time complexity that needs to be fixed, and frankly most of the notifier system should be eliminated.

If backward helps in real world cases I could commit this as a stopgap, but not sure how much performance gain is left after my commit.

The wm_event_do_notifiers part of that profile should be eliminated by rB4b46afae0951: WM: minor optimization for when there is a large number of notifiers.

For the rest, I think it's possible to write a Python script that is significantly worse in either case. If you send notifiers AAABBBCCC then backward will be faster. If you send ABACADAE then backward will be slower.

Fundamentally it's the time complexity that needs to be fixed, and frankly most of the notifier system should be eliminated.

If backward helps in real world cases I could commit this as a stopgap, but not sure how much performance gain is left after my commit.

Thanks Brecht, I'll test it and see if this is still relevant.

The wm_event_do_notifiers part of that profile should be eliminated by rB4b46afae0951: WM: minor optimization for when there is a large number of notifiers.

For the rest, I think it's possible to write a Python script that is significantly worse in either case. If you send notifiers AAABBBCCC then backward will be faster. If you send ABACADAE then backward will be slower.

Fundamentally it's the time complexity that needs to be fixed, and frankly most of the notifier system should be eliminated.

If backward helps in real world cases I could commit this as a stopgap, but not sure how much performance gain is left after my commit.

@Brecht Van Lommel (brecht)
From my simple testing (a script adding 10k simple objects) it seems like your patch completely removed this problem and is as fast as this idea of iterating backwards so this shouldn't be necessary to commit I guess.
I'll keep an eye out in future testing.. I'm trying to improve Blender's ability to handle many objects in all aspects.
I have another patch D9892 that maybe you're interesting in looking at to get some more inspiration as well.

I am going to close the patch if there is no longer relevant.