Page MenuHome

Weight & Vertex Paint: always respect edit mode hiding on faces.
ClosedPublic

Authored by Alexander Gavrilov (angavrilov) on Feb 20 2022, 8:50 PM.
Tokens
"Love" token, awarded by PrettyFireNOI7."Burninate" token, awarded by DiogoX2."Like" token, awarded by Draise."Love" token, awarded by verbal007."Love" token, awarded by Mets."Like" token, awarded by FkNWO."Like" token, awarded by Bit."Like" token, awarded by erickblender."Love" token, awarded by zanqdo."Doubloon" token, awarded by activemotionpictures."Like" token, awarded by RiggingDojo.

Details

Summary

In some cases it is necessary to be able to hide parts of the mesh
in order to paint certain poorly accessible areas. One worst case
example is the MakeHuman base mesh, which includes multiple
layers of clothing template layers in one mesh object with the body.
The Mask modifier, which is used to hide mesh parts for render,
doesn't work in weight paint, so a paint specific hiding method
is required.

Such method exists: mesh parts hidden in Edit mode are also
hidden in Weight Paint, but only if vertex or face selection is
enabled. However, it is not clear why hiding should be tied to
selection, and enabling selection has downsides like not being
able to select bones.

This patch makes the weight and vertex paint modes respect
edit mode hiding even without enabling selection, like in sculpt
mode. The change in behavior affects drawing and building paint
PBVH. Thus it affects brushes, but not menu operators like Smooth
or Normalize.

Diff Detail

Repository
rB Blender

Event Timeline

Alexander Gavrilov (angavrilov) requested review of this revision.Feb 20 2022, 8:50 PM
Alexander Gavrilov (angavrilov) created this revision.

Unless there is a good reason to have this exposed as an option I'd rather follow the behavior of other modes that respect hidden geometry (sculpt & edit-mode).

Could you check with riggers if there are problems with changing the current behavior?

I do use the hiding and masking functionality a lot, and I guess it is a little strange that whenever I want to use hiding, I am forced to use masking. I didn't mind it much since it's a couple quick keystrokes (V, A). This change would save me those couple keypresses, so 👍 to the patch.

The only downside of this is the same as in other modes: New user downloads file from interwebs, enters edit/sculpt/whatever mode, and if any geometry was hidden in edit mode, they'll go "where did my stuff go!?". This is still a point in favor of this patch, since at least if we have some awkward design at the core of Blender, it should be consistently awkward. So also 👍 to Campbell's suggestion that this doesn't need to be an option, since it's consistent with other modes.

That said, and this is scope creep/off topic: Having a user pref (off by default) that toggles whether edit-mode hiding masks ALL other modes, could fix this awkwardness at its core! :)

"
Pierrick Picaut
@pierrick_picaut
Replying to @Brad Clark (RiggingDojo) @ArminHalac @3D_director

That would definitely be a good move as it’s always a waste to have to switch to face /vertex selection.
Plus, this would allow to pick bones without charging mode again."

My vote is for consistent behavior as well, the more places things act the same, the easier it is to learn, to use, expectations are the same and muscle memory over how it works is standard.
So, thumbs up. It is strange to do the work to hid faces, then you jump to weight mode and are shown the entire mesh...I would rather it work like sculpt mode as proposed here.

I would rather it work like sculpt mode as proposed here.

The question asked is, should it always act as sculpt mode, or as an option like currently implemented by the patch. ;)

If you are going to keep it as an option to change how it works, then the default should be on like sculpt. If you removed the option to change it, then it should just default to sculpt. I like options though.

I'm also with Campbell. No need for a separate option, just make it respect edit mode hiding!

Removed the option, making the behavior change permanent.

Alexander Gavrilov (angavrilov) retitled this revision from Weight Paint: allow hiding faces without enabling selection. to Weight & Vertex Paint: always respect edit mode hiding on faces..Feb 22 2022, 9:10 PM
Alexander Gavrilov (angavrilov) edited the summary of this revision. (Show Details)
Campbell Barton (campbellbarton) requested changes to this revision.EditedMar 7 2022, 6:32 AM

In principle this seems fine, this could be committed with other changes to support hide/reveal in weight paint mode.

  • Expose "Show All" (Alt-H) bpy.ops.paint.hide_show(action='SHOW', area='ALL') (both shortcut and menu).
  • Expose tools Hide Bounding Box / Show Bounding Box.

Other features could be exposed too, but basic hide/reveal should be supported.
At the moment you could enter weight paint mode and have geometry vanish without an obvious way to reveal it.

While it is possible to enter edit-mode or enable face-selection, then reveal (Alt-H) this is quite hidden.

This revision now requires changes to proceed.Mar 7 2022, 6:32 AM

This patch was reviewed favorably in March 17 Animation & Rigging Module meeting. I offered to make a video demonstration of the patch for clarity:

Any progress on adding the hiding hotkeys Campbell requested?

Well, unfortunately I haven't been in a position to have much progress in anything for almost two months, and this will likely continue for a couple of weeks at least.

  • Implemented H and Alt-H shortcuts for vertex selection (it seems it was overlooked)
  • Allowed Alt-H without a selection mode for Vertex/Weight paint.

No menus were added, because Hide/Reveal for face mode existed but weren't in any menus, and thus there is no obvious UI design decision to follow.
The reveal operator is shared for all selection modes because of overlapping keymaps.

@Campbell Barton (campbellbarton) I believe your requested changes are all done?

Sybren A. Stüvel (sybren) requested changes to this revision.Jun 23 2022, 2:45 PM
Sybren A. Stüvel (sybren) added inline comments.
source/blender/editors/mesh/editface.c
629–630 ↗(On Diff #51264)

These declarations can move down to where they are used. Normally I wouldn't mind that much, but given that a is a rather undescriptive name (bonus points for finding a better one) it's important that the scope of the variable is as clear as possible.

Same for the function above.

source/blender/editors/sculpt_paint/paint_utils.c
763

That's not really a friendly name, and it suggests that it selects as well as hides. How about "Hide Vertices"?
The fact that this operates on selected vertices by default is so common that I feel it's fine to remove the "select" out of the name.

773

objects → vertices

781–784

Code style.

802

Same as above, not really a friendly name. "Reveal Faces/Vertices" is better IMO.

811–815

This should get some explanation, either here or in the operator description. Does this replace the current selection? Add to the selection? Something else?

This revision now requires changes to proceed.Jun 23 2022, 2:45 PM

Another option could be to add a toggle visibility to the vertex paint uilist in the vertex panel. It could show a visibility icon like in the outlines. This would save a lot of clicks as well and a user can hide multiple groups at once.

Best would be if this only executes when actually being in vertex weight paint mode. So returning to object or edit mode, this hiding will be disabled.

Here's a quick mockup

@Sybren A. Stüvel (sybren) I actually agree with all your comments, but I feel that you reviewed this as new code rather than copy & paste duplicates. Therefore fixing only this would cause inconsistency with surrounding code, which is a concern you didn't seem to address (i.e. choose to keep as is/fix only new code/fix old code too etc?).

source/blender/editors/mesh/editface.c
629–630 ↗(On Diff #51264)

This is based on copy & paste.

source/blender/editors/sculpt_paint/paint_utils.c
763

The meaning is "Vertex Select: Hide", i.e. hide operating in Vertex Select mode. Also, I did not design this naming scheme, it is direct copy & paste of Face Select Hide.

773

Same, this is verbatim copy & paste.

802

The naming scheme is not my idea, I only added "Vertex".

811–815

I did not change this code.

New code should use new code style. Only when moving should the old code be moved verbatim, so that it's clear it's only being moved without change in behaviour. And ideally it should have been refactored to get cleaned up *before* the move.

As for the current naming scheme, I don't like it at all. Also the "Hide Select Face" is weird.

The meaning is "Vertex Select: Hide", i.e. hide operating in Vertex Select mode.

To me that doesn't come across as such, from my perspective "Vertex Select Hide" is just three words thrown together.

Normally I would suggest fixing the naming scheme in a separate patch first, and then this patch can introduce the new operators with the new naming scheme. However, since we're about to move into BCON2 tomorrow, and I really want to see this patch landed before then, I'd say go ahead with the current naming scheme. A separate patch to address that is ok to land in BCON2.

Do take a look at the copied code, though. That should just adhere to the current standards.

Applied most of the requested changes.

Hans Goudey (HooglyBoogly) added inline comments.
source/blender/editors/mesh/editface.cc
563–566

Just passing through here (I have some refactors in this file to the way hiding flags are stored, which is why I'm interested). Could I suggest a simpler/more standard for loop syntax?

for (int i = 0; i < me->totvert; i++) {
  MVert &mvert = me->mvert[i];
  ...
}

Or if you wanted you could go a bit further:

for (const int i : IndexRange(me->totvert)) {

This makes the code easier to read, and makes my job easier when merging master into my refactor after this is committed ;)

Accepting, although hide/reveal should be in a menu, that was already an issue and can be addressed separately.

This revision is now accepted and ready to land.Jul 5 2022, 12:20 PM