Page MenuHome

Fix for crash when changing rendering devices
AbandonedPublic

Authored by William Leeson (leesonw) on Sep 15 2021, 3:44 PM.

Details

Summary

Stops draw calls when session is not ready to perform them.

Diff Detail

Repository
rB Blender

Event Timeline

From the task:

It appears the session is trying to draw the scene, while or after it has been cancelled. Since the session is cancelled from a different thread than it is drawn from and there are now locks around the drawing code.

I think the deeper issue is that different threads are cancelling the session and drawing at the same time. The only place where we call session->cancel() is BlenderSession::free_session, and that function deletes the session itself soon after, so it's not safe anyway.

I expect this is something to be fixed on the Blender side, ensuring we don't draw with RenderEngine on the main thread and delete it in the render thread at the same time.

Brecht Van Lommel (brecht) requested changes to this revision.Sep 15 2021, 7:24 PM
This revision now requires changes to proceed.Sep 15 2021, 7:24 PM

Seems you are correct the blender side of things seems to schedule the draw and the update from different threads

The draw stack is

#0  ccl::BlenderSession::draw (this=0x7fff830e6000, space_image=...)
    at /home/william/Source/blender-git/blender/intern/cycles/blender/blender_session.cpp:799
799	  if (!session || !session->scene) {
(gdb) bt
#0  ccl::BlenderSession::draw (this=0x7fff830e6000, space_image=...)
    at /home/william/Source/blender-git/blender/intern/cycles/blender/blender_session.cpp:799
#1  0x00000000062bb6be in ccl::draw_func (args=0x7fffc66431d0)
    at /home/william/Source/blender-git/blender/intern/cycles/blender/blender_python.cpp:286
#2  0x000000000e897144 in cfunction_call (func=0x7fffc6590a40, args=<optimized out>, kwargs=<optimized out>) at Objects/methodobject.c:548
#3  0x000000000e858e40 in _PyObject_MakeTpCall (tstate=0x7ffff4fe4d80, callable=0x7fffc6590a40, args=<optimized out>, 
    nargs=<optimized out>, keywords=0x0) at Objects/call.c:191
#4  0x0000000003794b10 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775812, args=0x7fffc49d19b0, 
    callable=0x7fffc6590a40, tstate=<optimized out>) at ./Include/cpython/abstract.h:116
#5  _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775812, args=0x7fffc49d19b0, callable=0x7fffc6590a40, 
    tstate=<optimized out>) at ./Include/cpython/abstract.h:103
#6  PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775812, args=0x7fffc49d19b0, callable=0x7fffc6590a40)
    at ./Include/cpython/abstract.h:127
#7  call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7ffff4fe4d80) at Python/ceval.c:5072
#8  _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3487
#9  0x000000000378ac3b in _PyEval_EvalFrame (throwflag=0, f=0x7fffc49d1800, tstate=0x7ffff4fe4d80) at ./Include/internal/pycore_ceval.h:40
#10 function_code_fastcall (tstate=0x7ffff4fe4d80, co=<optimized out>, args=<optimized out>, nargs=3, globals=<optimized out>)
    at Objects/call.c:330
#11 0x00000000037921ed in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x7fffc49c0a90, callable=0x7fffc65b3c10, 
    tstate=0x7ffff4fe4d80) at ./Include/cpython/abstract.h:118
#12 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7fffc49c0a90, callable=0x7fffc65b3c10)
    at ./Include/cpython/abstract.h:127
#13 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7ffff4fe4d80) at Python/ceval.c:5072
#14 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3487
#15 0x000000000378ac3b in _PyEval_EvalFrame (throwflag=0, f=0x7fffc49c0900, tstate=0x7ffff4fe4d80) at ./Include/internal/pycore_ceval.h:40
#16 function_code_fastcall (tstate=0x7ffff4fe4d80, co=<optimized out>, args=<optimized out>, nargs=3, globals=<optimized out>)
    at Objects/call.c:330
#17 0x000000000e85889f in PyVectorcall_Call (callable=0x7fffc65bc550, tuple=<optimized out>, kwargs=<optimized out>) at Objects/call.c:231
#18 0x00000000049452f2 in bpy_class_call (C=0x7ffff492f5c8, ptr=0x7fffffffd260, func=0x1512a280 <rna_RenderEngine_draw_func>, 
    parms=0x7fffffffd240) at /home/william/Source/blender-git/blender/source/blender/python/intern/bpy_rna.c:8550
#19 0x0000000004842d71 in engine_draw (engine=0x7fff830be008, context=0x7ffff492f5c8, depsgraph=0x7fffc8b7b608)
    at /home/william/Source/blender-git/blender/source/blender/makesrna/intern/rna_render.c:211
#20 0x00000000041b46d8 in external_draw_scene_do_image (UNUSED_vedata=0x7fffbc954c88)
    at /home/william/Source/blender-git/blender/source/blender/draw/engines/external/external_engine.c:391
#21 0x00000000041b4757 in external_draw_scene_do (vedata=0x7fffbc954c88)
    at /home/william/Source/blender-git/blender/source/blender/draw/engines/external/external_engine.c:417
#22 0x00000000041b4811 in external_draw_scene (vedata=0x7fffbc954c88)
    at /home/william/Source/blender-git/blender/source/blender/draw/engines/external/external_engine.c:440
#23 0x000000000416913a in drw_engines_draw_scene ()
--Type <RET> for more, q to quit, c to continue without paging--
    at /home/william/Source/blender-git/blender/source/blender/draw/intern/draw_manager.c:1087
#24 0x000000000416ba09 in DRW_draw_render_loop_2d_ex (depsgraph=0x7fffc8b7b608, region=0x7fffd6cc0c08, viewport=0x7fffbc954a08, 
    evil_C=0x7ffff492f5c8) at /home/william/Source/blender-git/blender/source/blender/draw/intern/draw_manager.c:2118
#25 0x000000000416a148 in DRW_draw_view (C=0x7ffff492f5c8)
    at /home/william/Source/blender-git/blender/source/blender/draw/intern/draw_manager.c:1495
#26 0x0000000005086c43 in image_main_region_draw (C=0x7ffff492f5c8, region=0x7fffd6cc0c08)
    at /home/william/Source/blender-git/blender/source/blender/editors/space_image/space_image.c:650
#27 0x000000000496de7a in ED_region_do_draw (C=0x7ffff492f5c8, region=0x7fffd6cc0c08)
    at /home/william/Source/blender-git/blender/source/blender/editors/screen/area.c:564
#28 0x0000000003df6120 in wm_draw_window_offscreen (C=0x7ffff492f5c8, win=0x7fffbc8e0388, stereo=false)
    at /home/william/Source/blender-git/blender/source/blender/windowmanager/intern/wm_draw.c:728
#29 0x0000000003df66eb in wm_draw_window (C=0x7ffff492f5c8, win=0x7fffbc8e0388)
    at /home/william/Source/blender-git/blender/source/blender/windowmanager/intern/wm_draw.c:877
#30 0x0000000003df6d23 in wm_draw_update (C=0x7ffff492f5c8)
    at /home/william/Source/blender-git/blender/source/blender/windowmanager/intern/wm_draw.c:1078
#31 0x0000000003df3291 in WM_main (C=0x7ffff492f5c8)
    at /home/william/Source/blender-git/blender/source/blender/windowmanager/intern/wm.c:654
#32 0x000000000379554a in main (argc=1, argv=0x7fffffffdf48) at /home/william/Source/blender-git/blender/source/creator/creator.c:558

and the sync stack is

#0  ccl::BlenderSync::reset (this=0x7fff6122a800, b_data=..., b_scene=...)
    at /home/william/Source/blender-git/blender/intern/cycles/blender/blender_sync.cpp:94
#1  0x00000000062c6386 in ccl::BlenderSession::reset_session (this=0x7fff830e6000, b_data=..., b_depsgraph=...)
    at /home/william/Source/blender-git/blender/intern/cycles/blender/blender_session.cpp:186
#2  0x00000000062bba7e in ccl::reset_func (args=0x7fffc5b064c0)
    at /home/william/Source/blender-git/blender/intern/cycles/blender/blender_python.cpp:367
#3  0x000000000e897144 in cfunction_call (func=0x7fffc6590b80, args=<optimized out>, kwargs=<optimized out>) at Objects/methodobject.c:548
#4  0x000000000e858e40 in _PyObject_MakeTpCall (tstate=0x7fff83009540, callable=0x7fffc6590b80, args=<optimized out>, 
    nargs=<optimized out>, keywords=0x0) at Objects/call.c:191
#5  0x0000000003794b10 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775811, args=0x7fffc49d31e8, 
    callable=0x7fffc6590b80, tstate=<optimized out>) at ./Include/cpython/abstract.h:116
#6  _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775811, args=0x7fffc49d31e8, callable=0x7fffc6590b80, 
    tstate=<optimized out>) at ./Include/cpython/abstract.h:103
#7  PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775811, args=0x7fffc49d31e8, callable=0x7fffc6590b80)
    at ./Include/cpython/abstract.h:127
#8  call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7fff83009540) at Python/ceval.c:5072
#9  _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3487
#10 0x000000000378ac3b in _PyEval_EvalFrame (throwflag=0, f=0x7fffc49d3040, tstate=0x7fff83009540) at ./Include/internal/pycore_ceval.h:40
#11 function_code_fastcall (tstate=0x7fff83009540, co=<optimized out>, args=<optimized out>, nargs=3, globals=<optimized out>)
    at Objects/call.c:330
#12 0x00000000037921ed in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x7fffc49d81e0, callable=0x7fffc65b3d30, 
    tstate=0x7fff83009540) at ./Include/cpython/abstract.h:118
#13 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x7fffc49d81e0, callable=0x7fffc65b3d30)
    at ./Include/cpython/abstract.h:127
#14 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x7fff83009540) at Python/ceval.c:5072
#15 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3487
#16 0x000000000378ac3b in _PyEval_EvalFrame (throwflag=0, f=0x7fffc49d8040, tstate=0x7fff83009540) at ./Include/internal/pycore_ceval.h:40
#17 function_code_fastcall (tstate=0x7fff83009540, co=<optimized out>, args=<optimized out>, nargs=3, globals=<optimized out>)
    at Objects/call.c:330
#18 0x000000000e85889f in PyVectorcall_Call (callable=0x7fffc65bc430, tuple=<optimized out>, kwargs=<optimized out>) at Objects/call.c:231
#19 0x00000000049452f2 in bpy_class_call (C=0x7ffff492f5c8, ptr=0x7fff81737d30, func=0x15129e60 <rna_RenderEngine_update_func>, 
    parms=0x7fff81737d10) at /home/william/Source/blender-git/blender/source/blender/python/intern/bpy_rna.c:8550
#20 0x0000000004842b88 in engine_update (engine=0x7fff830be008, bmain=0x7fffc7430608, depsgraph=0x7fff83017508)
    at /home/william/Source/blender-git/blender/source/blender/makesrna/intern/rna_render.c:160
#21 0x000000000e0e0a50 in engine_render_view_layer (re=0x7fffbc946408, engine=0x7fff830be008, view_layer_iter=0x7fff83008008, 
    use_engine=true, use_grease_pencil=true) at /home/william/Source/blender-git/blender/source/blender/render/intern/engine.c:880
#22 0x000000000e0e0f62 in RE_engine_render (re=0x7fffbc946408, do_all=false)
    at /home/william/Source/blender-git/blender/source/blender/render/intern/engine.c:1004
#23 0x000000000e0e835d in do_render_engine (re=0x7fffbc946408)
--Type <RET> for more, q to quit, c to continue without paging--
    at /home/william/Source/blender-git/blender/source/blender/render/intern/pipeline.c:1061
#24 0x000000000e0e884f in do_render_compositor (re=0x7fffbc946408)
    at /home/william/Source/blender-git/blender/source/blender/render/intern/pipeline.c:1190
#25 0x000000000e0e9470 in do_render_full_pipeline (re=0x7fffbc946408)
    at /home/william/Source/blender-git/blender/source/blender/render/intern/pipeline.c:1457
#26 0x000000000e0ea50e in RE_RenderFrame (re=0x7fffbc946408, bmain=0x7fffc7430608, scene=0x7fff99059008, single_layer=0x0, 
    camera_override=0x0, frame=132, write_still=false)
    at /home/william/Source/blender-git/blender/source/blender/render/intern/pipeline.c:1860
#27 0x00000000062a0dd3 in render_startjob (rjv=0x7fffd6cc0f88, stop=0x7fffbc24f1c4, do_update=0x7fffbc24f1c2, progress=0x7fffbc24f1c8)
    at /home/william/Source/blender-git/blender/source/blender/editors/render/render_internal.c:651
#28 0x0000000003e15fd7 in do_job_thread (job_v=0x7fffbc24f148)
    at /home/william/Source/blender-git/blender/source/blender/windowmanager/intern/wm_jobs.c:393
#29 0x000000000fe1207b in tslot_thread_start (tslot_p=0x7fffbcf35b08)
    at /home/william/Source/blender-git/blender/source/blender/blenlib/intern/threads.cc:231
#30 0x00007ffff7f8f299 in start_thread () from /lib64/libpthread.so.0
#31 0x00007ffff7b6c353 in clone () from /lib64/libc.so.6

Unfortunately, there does not seem to be a logical point to synchronize between these 2 threads any ideas?

One of the ideas I have is that the RE_engine_is_rendering should only return true only if the engine's render() callback is in process. Meaning, it can not use RE_ENGINE_RENDERING, and would need to introduce some other flag indicating that render() is running.

The bug this was trying to fix is fixed by @Sergey Sharybin (sergey) in D12521