Page MenuHome

Crash by hiting "tab" after sculpting with dyntopo.
Closed, ResolvedPublicBUG

Description

System Information
Operating system: Win 10
Graphics card: 950m
Working with mouse. Graphics software updated.

Blender Version
Broken: d525c76003b3
Worked: (optional)

Short description of error
If you click "tab" after any change of mesh in dyntopo (relative) mode, blender crash (more often if you add detail). However if you manualy change mode in bottom. There is hight chanse to avoid crash.

I have trouble with ctrl-z too, even if blender somehow manage to not crash. He have tendency to broke after some edit of mesh without dyntopo and clicking a few times ctrl-z. Sorry i can't give much more information obout ctrl-z. It generaly starts glitching in my other projects as well over the last .. month ?

Exact steps for others to reproduce the error
Just folow short video.


Event Timeline

George Vogiatzis (Gvgeo) lowered the priority of this task from 90 to 50.

when dyntopo finish, there is a problem in build_pbvh_from_regular_mesh .
When it sends BKE_pbvh_apply_vertCos(pbvh, v_cos, totvert);

void BKE_pbvh_apply_vertCos(PBVH *pbvh, float (*vertCos)[3], const int totvert)
{
    if (totvert != pbvh->totvert) {
        BLI_assert(!"PBVH: Given deforming vcos number does not natch PBVH vertex number!");
 	blender.exe!issue_debug_notification(const wchar_t * const message) Line 28	C++
 	blender.exe!__acrt_report_runtime_error(const wchar_t * message) Line 154	C++
 	blender.exe!abort() Line 61	C++
 	blender.exe!BKE_pbvh_apply_vertCos(PBVH * pbvh, float[3] * vertCos, const int totvert) Line 2328	C
 	blender.exe!build_pbvh_from_regular_mesh(Object * ob, Mesh * me_eval_deform) Line 1443	C
 	blender.exe!BKE_sculpt_object_pbvh_ensure(Depsgraph * depsgraph, Object * ob) Line 1499	C
>	blender.exe!BKE_sculpt_update_mesh_elements(Depsgraph * depsgraph, Scene * scene, Sculpt * sd, Object * ob, bool need_pmap, bool need_mask) Line 1222	C
 	blender.exe!sculpt_update_after_dynamic_topology_toggle(Depsgraph * depsgraph, Scene * scene, Object * ob) Line 5623	C
 	blender.exe!sculpt_dynamic_topology_disable_ex(Depsgraph * depsgraph, Scene * scene, Object * ob, SculptUndoNode * unode) Line 5744	C
 	blender.exe!sculpt_dynamic_topology_disable_with_undo(Depsgraph * depsgraph, Scene * scene, Object * ob) Line 5763	C
 	blender.exe!sculpt_dynamic_topology_toggle_exec(bContext * C, wmOperator * UNUSED_op) Line 5791	C
 	blender.exe!sculpt_dynamic_topology_toggle_invoke(bContext * C, wmOperator * op, const wmEvent * UNUSED_event) Line 5901	C
 	blender.exe!wm_operator_invoke(bContext * C, wmOperatorType * ot, wmEvent * event, PointerRNA * properties, ReportList * reports, const bool poll_only, bool use_last_properties) Line 1407	C
 	blender.exe!wm_operator_call_internal(bContext * C, wmOperatorType * ot, PointerRNA * properties, ReportList * reports, const short context, const bool poll_only, wmEvent * event) Line 1621	C
 	blender.exe!WM_operator_name_call_ptr(bContext * C, wmOperatorType * ot, short context, PointerRNA * properties) Line 1670	C
 	blender.exe!ui_apply_but_funcs_after(bContext * C) Line 809	C
 	blender.exe!ui_handler_region_menu(bContext * C, const wmEvent * event, void * UNUSED_userdata) Line 10146	C
 	blender.exe!wm_handler_ui_call(bContext * C, wmEventHandler_UI * handler, const wmEvent * event, int always_pass) Line 608	C
 	blender.exe!wm_handlers_do_intern(bContext * C, wmEvent * event, ListBase * handlers) Line 2659	C
 	blender.exe!wm_handlers_do(bContext * C, wmEvent * event, ListBase * handlers) Line 2844	C
 	blender.exe!wm_event_do_handlers(bContext * C) Line 3212	C
 	blender.exe!WM_main(bContext * C) Line 420	C
 	blender.exe!main(int argc, const unsigned char * * UNUSED_argv_c) Line 502	C
 	[External Code]
Bastien Montagne (mont29) raised the priority of this task from 50 to High.

@Paweł Żukowicz (zukson) Please follow our submission template and guidelines, also read these tips about bug reports, and make a complete, valid bug report, with required info, precise description of the issue (only ONE issue per report!), precise steps to reproduce it, small and simple .blend and/or other files to do so if needed, etc.
Videos and/or links to external sites etc. are not acceptable as bug report (they can be provided as additional information only).

This is a complex model, I cannot reproduce it from scratch with a dummy cube it'd seem…

@George Vogiatzis (Gvgeo) Please do not assign randomly reports to people, that is not especially my area…

@Sergey Sharybin (sergey) issue here is that in BKE_sculpt_update_mesh_elements() (which is called when switching dyntopo off just before leaving sculpt mode), the orig mesh of the object is updated from final state of sculpt session, while the evaluated one is still based on mesh before the sculpt session (I think, at least that's what its number of verts suggests).
Thing is, I tried to force-update depsgraph (hackish DEG_on_visible_update() call in sculpt_update_after_dynamic_topology_toggle() before calling BKE_sculpt_update_mesh_elements(), but am still getting an me_eval with wrong (old) number of verts.
Assigning to you, at least to get some hits, am feeling a bit lost in that code…
Note that bug can be triggered as well by just trying to disable dyntopo after sculpting a bit, no need to go to Edit mode.

@Bastien Montagne (mont29) Certainly not random, I saw commit rB934b9e80 that catches the problem and rB3e86bb2d.

@Bastien Montagne (mont29), i don't think this is anything to do with depsgraph or evaluation, this is something to do with sculpt mode attempting to do deformed sculpt on a dynamic topology. This isn't going to work: dynamic topology only exists in sculpt mode and is not accessible in modifier stack.

Basically, BKE_pbvh_apply_vertCos() is not to be used when dynamic topology is used. Not sure why it's fine when you enable dynamic topology and why is it only becoming a problem later on.

P.S. Dynamic topology was never supported for deformed sculpting, you can see this in a simple attached file. Open in 2.79, go to sculpt mode, enable dynamic topology.

@Bastien Montagne (mont29), bad news is: got slightly diverged into a wrong direction of investigation. Good news is, now i know what's going on and have some patch to be discussed.

Basically, we indeed need to ensure object is fully evaluated prior to re-creating the PBVH. That puts all the required bits into a nice consistent state.

We can not use DEG_on_visible_update() since that only tags for updates, it does not to actual update.
The naming is confusing, comes from some legacy history, and is probably be changed.

Here is a patch which works, but which probably required extra pair of eyes and brain to go through:

1diff --git a/source/blender/editors/include/ED_object.h b/source/blender/editors/include/ED_object.h
2index ed0298cce1f..a4c68c2c5ad 100644
3--- a/source/blender/editors/include/ED_object.h
4+++ b/source/blender/editors/include/ED_object.h
5@@ -183,7 +183,8 @@ void ED_object_sculptmode_enter_ex(struct Main *bmain,
6 const bool force_dyntopo,
7 struct ReportList *reports);
8 void ED_object_sculptmode_enter(struct bContext *C, struct ReportList *reports);
9-void ED_object_sculptmode_exit_ex(struct Depsgraph *depsgraph,
10+void ED_object_sculptmode_exit_ex(struct Main *bmain,
11+ struct Depsgraph *depsgraph,
12 struct Scene *scene,
13 struct Object *ob);
14 void ED_object_sculptmode_exit(struct bContext *C);
15diff --git a/source/blender/editors/object/object_modes.c b/source/blender/editors/object/object_modes.c
16index cc28ceaf95b..0f0d09c610b 100644
17--- a/source/blender/editors/object/object_modes.c
18+++ b/source/blender/editors/object/object_modes.c
19@@ -276,7 +276,7 @@ static bool ed_object_mode_generic_exit_ex(struct Main *bmain,
20 if (only_test) {
21 return true;
22 }
23- ED_object_sculptmode_exit_ex(depsgraph, scene, ob);
24+ ED_object_sculptmode_exit_ex(bmain, depsgraph, scene, ob);
25 }
26 }
27 else if (ob->mode & OB_MODE_POSE) {
28diff --git a/source/blender/editors/sculpt_paint/sculpt.c b/source/blender/editors/sculpt_paint/sculpt.c
29index 9ba7561b18b..23656737153 100644
30--- a/source/blender/editors/sculpt_paint/sculpt.c
31+++ b/source/blender/editors/sculpt_paint/sculpt.c
32@@ -60,6 +60,7 @@
33 #include "BKE_pbvh.h"
34 #include "BKE_pointcache.h"
35 #include "BKE_report.h"
36+#include "BKE_scene.h"
37 #include "BKE_screen.h"
38 #include "BKE_subsurf.h"
39
40@@ -5672,10 +5673,8 @@ void sculpt_dynamic_topology_enable_ex(Depsgraph *depsgraph, Scene *scene, Objec
41 *
42 * If 'unode' is given, the BMesh's data is copied out to the unode
43 * before the BMesh is deleted so that it can be restored from */
44-void sculpt_dynamic_topology_disable_ex(Depsgraph *depsgraph,
45- Scene *scene,
46- Object *ob,
47- SculptUndoNode *unode)
48+void sculpt_dynamic_topology_disable_ex(
49+ Main *bmain, Depsgraph *depsgraph, Scene *scene, Object *ob, SculptUndoNode *unode)
50 {
51 SculptSession *ss = ob->sculpt;
52 Mesh *me = ob->data;
53@@ -5739,19 +5738,24 @@ void sculpt_dynamic_topology_disable_ex(Depsgraph *depsgraph,
54 BKE_particlesystem_reset_all(ob);
55 BKE_ptcache_object_reset(scene, ob, PTCACHE_RESET_OUTDATED);
56
57+ DEG_id_tag_update(&ob->id, ID_RECALC_GEOMETRY);
58+ BKE_scene_graph_update_tagged(depsgraph, bmain);
59+
60 /* Refresh */
61 sculpt_update_after_dynamic_topology_toggle(depsgraph, scene, ob);
62 }
63
64 void sculpt_dynamic_topology_disable(bContext *C, SculptUndoNode *unode)
65 {
66+ Main *bmain = CTX_data_main(C);
67 Depsgraph *depsgraph = CTX_data_depsgraph(C);
68 Scene *scene = CTX_data_scene(C);
69 Object *ob = CTX_data_active_object(C);
70- sculpt_dynamic_topology_disable_ex(depsgraph, scene, ob, unode);
71+ sculpt_dynamic_topology_disable_ex(bmain, depsgraph, scene, ob, unode);
72 }
73
74-static void sculpt_dynamic_topology_disable_with_undo(Depsgraph *depsgraph,
75+static void sculpt_dynamic_topology_disable_with_undo(Main *bmain,
76+ Depsgraph *depsgraph,
77 Scene *scene,
78 Object *ob)
79 {
80@@ -5759,7 +5763,7 @@ static void sculpt_dynamic_topology_disable_with_undo(Depsgraph *depsgraph,
81 if (ss->bm) {
82 sculpt_undo_push_begin("Dynamic topology disable");
83 sculpt_undo_push_node(ob, NULL, SCULPT_UNDO_DYNTOPO_END);
84- sculpt_dynamic_topology_disable_ex(depsgraph, scene, ob, NULL);
85+ sculpt_dynamic_topology_disable_ex(bmain, depsgraph, scene, ob, NULL);
86 sculpt_undo_push_end();
87 }
88 }
89@@ -5779,6 +5783,7 @@ static void sculpt_dynamic_topology_enable_with_undo(Depsgraph *depsgraph,
90
91 static int sculpt_dynamic_topology_toggle_exec(bContext *C, wmOperator *UNUSED(op))
92 {
93+ Main *bmain = CTX_data_main(C);
94 Depsgraph *depsgraph = CTX_data_depsgraph(C);
95 Scene *scene = CTX_data_scene(C);
96 Object *ob = CTX_data_active_object(C);
97@@ -5787,7 +5792,7 @@ static int sculpt_dynamic_topology_toggle_exec(bContext *C, wmOperator *UNUSED(o
98 WM_cursor_wait(1);
99
100 if (ss->bm) {
101- sculpt_dynamic_topology_disable_with_undo(depsgraph, scene, ob);
102+ sculpt_dynamic_topology_disable_with_undo(bmain, depsgraph, scene, ob);
103 }
104 else {
105 sculpt_dynamic_topology_enable_with_undo(depsgraph, scene, ob);
106@@ -6151,7 +6156,7 @@ void ED_object_sculptmode_enter(struct bContext *C, ReportList *reports)
107 ED_object_sculptmode_enter_ex(bmain, depsgraph, scene, ob, false, reports);
108 }
109
110-void ED_object_sculptmode_exit_ex(Depsgraph *depsgraph, Scene *scene, Object *ob)
111+void ED_object_sculptmode_exit_ex(Main *bmain, Depsgraph *depsgraph, Scene *scene, Object *ob)
112 {
113 const int mode_flag = OB_MODE_SCULPT;
114 Mesh *me = BKE_mesh_from_object(ob);
115@@ -6177,7 +6182,7 @@ void ED_object_sculptmode_exit_ex(Depsgraph *depsgraph, Scene *scene, Object *ob
116 /* Dynamic topology must be disabled before exiting sculpt
117 * mode to ensure the undo stack stays in a consistent
118 * state */
119- sculpt_dynamic_topology_disable_with_undo(depsgraph, scene, ob);
120+ sculpt_dynamic_topology_disable_with_undo(bmain, depsgraph, scene, ob);
121
122 /* store so we know to re-enable when entering sculpt mode */
123 me->flag |= ME_SCULPT_DYNAMIC_TOPOLOGY;
124@@ -6199,11 +6204,12 @@ void ED_object_sculptmode_exit_ex(Depsgraph *depsgraph, Scene *scene, Object *ob
125
126 void ED_object_sculptmode_exit(bContext *C)
127 {
128+ Main *bmain = CTX_data_main(C);
129 Depsgraph *depsgraph = CTX_data_depsgraph(C);
130 Scene *scene = CTX_data_scene(C);
131 ViewLayer *view_layer = CTX_data_view_layer(C);
132 Object *ob = OBACT(view_layer);
133- ED_object_sculptmode_exit_ex(depsgraph, scene, ob);
134+ ED_object_sculptmode_exit_ex(bmain, depsgraph, scene, ob);
135 }
136
137 static int sculpt_mode_toggle_exec(bContext *C, wmOperator *op)
138@@ -6225,7 +6231,7 @@ static int sculpt_mode_toggle_exec(bContext *C, wmOperator *op)
139 }
140
141 if (is_mode_set) {
142- ED_object_sculptmode_exit_ex(depsgraph, scene, ob);
143+ ED_object_sculptmode_exit_ex(bmain, depsgraph, scene, ob);
144 }
145 else {
146 ED_object_sculptmode_enter_ex(bmain, depsgraph, scene, ob, false, op->reports);
147diff --git a/source/blender/editors/sculpt_paint/sculpt_intern.h b/source/blender/editors/sculpt_paint/sculpt_intern.h
148index be4ab5809e4..b29ca1b4dd7 100644
149--- a/source/blender/editors/sculpt_paint/sculpt_intern.h
150+++ b/source/blender/editors/sculpt_paint/sculpt_intern.h
151@@ -33,6 +33,7 @@
152
153 #include "BKE_pbvh.h"
154
155+struct Main;
156 struct KeyBlock;
157 struct Object;
158 struct SculptOrigVertData;
159@@ -58,7 +59,8 @@ void sculpt_dynamic_topology_enable_ex(struct Depsgraph *depsgraph,
160 struct Scene *scene,
161 struct Object *ob);
162
163-void sculpt_dynamic_topology_disable_ex(struct Depsgraph *depsgraph,
164+void sculpt_dynamic_topology_disable_ex(struct Main *bmain,
165+ struct Depsgraph *depsgraph,
166 struct Scene *scene,
167 struct Object *ob,
168 struct SculptUndoNode *unode);

P.S. Noticed after submitting the patch. The recalc flag is also to include ID_RECALC_PSYS_RESET. BKE_ptcache_object_reset() might be fine, since the cache is supposed to be shared.

@Bastien Montagne (mont29) HI sory but how ?-? this could be posible to describe this in other way (i already reported some bugs when i use video as addisional help) ? I mention CTRL-Z becouse i found that conected with this error. Short video max20 sek with clear view of what i'm doing from opening file to error without any unnessesary steps. Of course i try recreate that with simple cube or ecosphere bit i failed so i have to report with this complicated to analize model. I WILL be better next time with more precise description of the error. Thank you all for your work.

Hello,
I read that the issue is "resolved". What is process from here, do we wait for a future update release for blender which will include the patch? I just tried and the issue is still happening.
Thanks and regards,

Hard to tell why this the issue is still happening for you since we don't have information about your build hash, platform and things like that.

Current builds which are available on builder.blender.org should have this issue resolved. If the bug is still happening for you, make a new report following the bug report guidelines.

Anyway, usual process is simple:

  • Wait for the next buildbot round to give new builds
  • Test the new builds
  • Report a new bug if the issue is still happening for you (there could be multiple code aspects even for the similar-sounding bug. Having those reported separately helps tracking down history).
  • Be happy if the issue is solved.

Indeed, my build was outdated. Downloading the latest build solved the issue.
Thanks !