Page MenuHome

Fix T95185: Invalid normals after undo in sculpt mode
AbandonedPublic

Authored by Hans Goudey (HooglyBoogly) on Jan 27 2022, 11:16 PM.

Details

Summary

Going back to rB5505697ac508c02b8a2e196c there was this weird
behavior that vertex normals were saved in the sculpt undo stack
but then freed a moment later.

After the vertex normal refactor, normals could be invalid after undo
in sculpt mode. I'm not confident about how it worked before though,
unfortunately. My current guess is that some MVert array was shared,
and the normals were calculated elsewhere.

To fix the invalid normal issue, I removed the strange code that freed
the normals, and copied them back to the current array when restoring
the undo step.


I really don't feel confident about this solution. I would have thought
that tagging every restored vertex with ME_VERT_PBVH_UPDATE
would cause their normals to recalculate properly anyway, but in my
testing that didn't work. I'm not sure why at this point.

I'm posting the patch because since it was my patch that caused
the problem I have a responsibility to take a stab at fixing it. But
it's my hope that someone more familiar with this area can say
"No no no, you're doing this wrong" or something.

Diff Detail

Event Timeline

This is caused by partial normal updates accumulating into an array of existing normals.
Before rBd9c6ceb3b88b, normals were accumulated into a newly cleared array.

This wasn't just an issue with undo, regular normal updates also had this problem, it was just less noticeable.

Submitted alternate patch D13975.

D13975 is much much better!