Page MenuHome

Fix T84453: Crash bezier curves transfrom
AbandonedPublic

Authored by Falk David (filedescriptor) on Jan 6 2021, 2:27 PM.

Details

Summary

When doing any proporional editing transformation operation on a bezier
curve that has hidden handles, Blender will crash (after save).

This bug is caused by the fact that during the initialization phase of the
TransData of the curve, hidden points are skipped in the counting.
But when the TransData is filled in, the hidden points are included
if proportional editing is turned on. Consequently the td and tail
pointers are missaligned (incremented) and ultimately cause a
heap-buffer-overflow.
The fix includes the hidden points in the counting to allow the flag
TD_NOTCONNECTED to be set for the hidden points. A potentially
better solution might be to find another way of handling the hidden
points while transforming.

Diff Detail

Repository
rB Blender
Branch
T84453 (branched from master)
Build Status
Buildable 12016
Build 12016: arc lint + arc unit

Event Timeline

Falk David (filedescriptor) requested review of this revision.Jan 6 2021, 2:27 PM

I may be missing some details, but at first glance, this solution seems strange because:

  • hidden elements are not affected by the transformation, so should they really be counted and allocated as TransData?
  • With this patch, Nurb-type curves still do not count hidden elements. Usually the code for CU_NURBS reflects that for CU_BEZIER, so it might be good to describe why they got "different".

(My impression is that td++; and that tail++; should be removed).

@Germano Cavalcante (mano-wii) Yes, in fact my first solution was to just remove the

else if (is_prop_edit && head != tail) {
  tail->flag |= TD_NOTCONNECTED;
  td++;
  tail++;
}

part. I thought this would work just fine as well. The problem is that now we cannot set the TD_NOTCONNECTED flag, which calc_distanceCurveVerts uses. Just removing td++; and tail++; also won't work because the calc_distanceCurveVerts function expects the hidden handle to have that flag set. But I think I would agree that a better solution is to just remove the else if and rewrite the calc_distanceCurveVerts function so that we don't need the hidden handles anymore.

Reanalyzing the code, I don't think TD_NOTCONNECTED is making any difference because, if I'm not mistaken, hidden elements are not considered selected and therefore are not added to the queue in calc_distanceCurveVerts.
I actually think that this TD_NOTCONNECTED flag and the td->dist member are amenable to future removal as td->rdist with the value FLT_MAX is enough to indicate what is not affected proportional edit and the Connected Only option apparently cannot be changed during the operation.

If you don't mind, I plan to edit this patch to focus on an alternative solution. (I still think td++; and that tail++; should be removed).

Thanks for the patch.
I submitted a different solution (rB2d3f96cace6d: Fix T84453: Crash bezier curves transfrom),
If you find any problems, feel free to edit this patch or create another review ;)