Not sure what to do anyway, unless you can show us that Blender is responsible for that.

It is crashing immediately after RENDERED mode is turned off. No addon code is executed on this event - therefore it's something in Blender?

Did you try doing that in exact same python directly (without using Blender's interpreter)?

Sure, threads with sleep(0) works fine in pure python.

Not sure to understand the point of using sleep(0)

Doesn't matter for me, personally, as anything non-zero passed to sleep seemingly fixes the issue.
But there's something wrong, this code should not crash.

This is not related to my change actually, but rather specific to how threading works in Python.

Basically, before my change the shading mode was changing via python script, so it was a stacktrace like:

• C code
• Python code
• C code

From the C side we do use BPy_BEGIN_ALLOW_THREADS /BPy_END_ALLOW_THREADS to ensure we don't run into threading issues with render engine API. This works fine for Cycles (which doesn't use Python threads), but gives issues in this particular case. Seems call of BPy_BEGIN_ALLOW_THREADS causes race condition with the thread which is still spinning from the python side.

Those begin/end blocks were added in rB5036ac6. Not yet sure whether it's specific to our BPY_thread_save()/BPY_thread_restore() implementation or not.

With the old code it was still possible to crash blender by changing shading using menu instead of the shortcut.

Looks like python threads are just doomed to crash when Shading mode is toggled. It's not just about sleep/wait.
E.g. replacing sleep(0) in the example code with simple with lock: pass makes it crash immediately on second Shift-Z.

Is there a possible workaround? Does addon code have any chance to shutdown it's threads before shading mode is toggled? Or defer the toggle so it doesn't happens immediately when Shift-Z is pressed?

Hmpff… the main issue is that using python threads inside of Blender is risky at best, it’s only reasonably safe when py threads are fully outside of Blender context (i.e. do not use Blender data, do not call Blender API, are not called by Blender, etc.). I *think* the issue here is that renderengine callabacks can be called from a job, i.e. from non-main Blender thread, if that’s the case those callbacks should definitively *never* use any python threading.

Further more, due to main lock in python, pythreads are usually not interesting for performances-driven parallelization, one should rather use processes for that.

Am tempted to consider this as known limitation for now, do you have any reason to actually use pythreads here, and not e.g. concurrent.future with a process pool? On the other hand, if you only want some kind of async update, then I would recommend looking at asyncio, even though its usage remains a bit dodgy in Blender currently…

Yes, I'm aware that Blender context should not be used in a thread. But my examples I think are completely avoiding it. If I'm not mistaken ;)

The reason we are using python threads - is convenience. We have a third-party dll that has rendering capabilities and thread is essentially needed to call a (blocking) 'render' function asynchronously. Your guess is right. This requires a thread. Everything else - is suboptimal. Native thread will add substantial complexity(C++ code), same with interprocess communication. And asyncio won't help - threads will need to be used there too for a blocking task(which we have here).

Am tempted to consider this as known limitation for now,

Worked fine in 2.77!!! From my perspective)

I *think* the issue here is that renderengine callabacks can be called from a job, i.e. from non-main Blender thread, if that’s the case those callbacks should definitively *never* use any python threading.

Are you saying that the problem might be that the thread is started from an engine callback?
Is it a viable workaround then - call a custom operator while in view_update and start that thread from the operator's execute?

Seems like ANY python thread might crash when RENDERED is being turned off. In my test I've simply added an operator that spawns a thread. Not using a renderengine callback(callbacks are empty). It crashes.