The old mask brush implementation was adding the brush value to the
previous vertex mask value and clamping the result. This leads to
visible artifacts in the mask gradient as the value approaches 0 or 1,
so it was not possible to paint a smooth mask with this brush.
Now we are also multiplying by the previous mask value before clamping,
fixing all those gradient artifacts.
Before:

After: