Page MenuHome

Mesh: Move UV layers to generic attributes
ClosedPublic

Authored by Martijn Versteegh (Baardaap) on Mar 16 2022, 11:14 PM.
Tags
None
Tokens
"Love" token, awarded by thainayu."Love" token, awarded by lictex_."Love" token, awarded by kursadk."Love" token, awarded by Takanu."Burninate" token, awarded by TitusLVR."Love" token, awarded by Dangry."Love" token, awarded by aras_p."Love" token, awarded by Bit."Like" token, awarded by Saseska.

Details

Summary

Currently the MLoopUV struct stores UV coordinates and flags related
to editing UV maps in the UV editor. This patch changes the coordinates
to use the generic 2D vector type, and moves the flags into three
separate boolean attributes. This follows the design in T95965, with
the ultimate intention of simplifying code and improving performance.

Importantly, the change allows exporters and renderers to use UVs
"touched" by geometry nodes, which only creates generic attributes.
It also allows geometry nodes to create UV maps from scratch,
though only with the Store Named Attribute node.

The new design considers any 2D vector attribute on the corner domain
to be a UV map. In the future, they might be distinguished from regular
2D vectors with attribute metadata, which may be helpful because they
are often interpolated differently.

Most of the code changes deal with passing around UV BMesh custom data
offsets and tracking the boolean "sublayers". The boolean layers are
use the following prefixes for attribute names: vert selection: .vs.,
edge selection: .es., pinning: .pn.. Currently these are short to
avoid using up the maximum length of attribute names. To accommodate for
these 4 extra characters, the name length limit is enlarged to 68 bytes,
while the maximum user settable name length is still 64 bytes.

Unfortunately Python/RNA API access to the UV flag data becomes slower.
Accessing the boolean layers directly is be better for performance in general.

Like the other mesh SoA refactors, backward and forward compatibility
aren't affected, and won't be changed until 4.0. We pay for that by
making mesh reading and writing more expensive with conversions.

Resolves T85962


Branch: refactor-mesh-uv-map-generic

Diff Detail

Repository
rB Blender
Branch
uvasattrib2 (branched from master)
Build Status
Buildable 21720
Build 21720: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
source/blender/makesdna/DNA_customdata_types.h
51

In the gui it's still capped at 64 characters (at least it *should* be if I didn't make any mistakes since...) . Though I must admit I don't have a clear mental model of all the ways people could set these names.

I'll have a better look at that. We do need to have this constant available in the source though, so enforcing the max length of 64 should not be based on this constant.

It's an ugly situation. And by far my least favourite part of the patch. But all other solutions of linking attributes I came up with were a lot more complicated.

source/blender/makesrna/intern/rna_mesh.c
1466–1468

I agree some documentation would be nice. However.. I though a doc-string was python documentation? This function is not callable from python. I think a comment would suffice?

1486

Will do.

This is indeed a murky area of the python api, with the python objects often having raw pointers pointing inside the customdata layers. The original ptr->data can easily be invalidated as well if the uv layer is reallocated in some way between getting the BPY_MLoopUV object and using it. :-(

1504

Hans is not gonna like it ;-) , but indeed better to be consistent.

1799

I added back in the report.
btw, wouldn't it be better to change it to

"UV map '%s" not found"

as Texture Layer isn't really the correct term anymore I think?

source/blender/modifiers/intern/MOD_array.cc
676 ↗(On Diff #59067)

Will change.

Some (lots of, actually) files were moved to C++ *after* I started the patch. I suspect this is one of them.

source/blender/python/bmesh/bmesh_py_types_customdata.c
1240

You are completely right, I somehow had confused it with it as Bpy_MLoopUV.

source/blender/python/bmesh/bmesh_py_types_meshdata.c
39–41

I must admit I didn't understand much of the python API stuff when creating this. And it's still not 100% clear to me how everything fits together. So this code is the result of progressive search&replace from the original code to keep the same outcome for each change I made.

It could undoubtedly be more efficient.

73

Will try to find out how to do that...

Martijn Versteegh (Baardaap) marked 7 inline comments as done.Jan 5 2023, 2:08 PM
Martijn Versteegh (Baardaap) marked 3 inline comments as done.Jan 5 2023, 2:39 PM
Martijn Versteegh (Baardaap) added inline comments.
source/blender/blenkernel/intern/attribute.cc
878 ↗(On Diff #59067)

Oops, probably written at night when I couldn't decide between <= -5 or < -4

source/blender/bmesh/intern/bmesh_interp.h
69–73 ↗(On Diff #59067)

I don't really agree, but will change.

source/blender/editors/include/ED_uvedit.h
89–90

Oh. hm. it's C, so no pass by const reference.

Still... isn't this something better left to the compiler these days?

source/blender/makesdna/DNA_customdata_types.h
51

Oh, hm. I now see your other comment that it's apparently not (anymore?) capped to 64 in the gui.
I'll need to look into that.

source/blender/python/bmesh/bmesh_py_types_meshdata.c
39–41

(urg, this should go *above the 'NOTE:' comment as it's a replay to the one above that. But as I wrote it later phab decides to put it below everything)

They should never be NULL when accessed/needed but created on-demand eventually. So probably I should add more asserts/ensure() calls.

My first plan was to have the lazy-allocation of the bool layers also work in BMesh (like it works in Mesh). But since allocating CustomData layers on demand in BMesh is
a) expensive
b) re-allocates layers/ rebuilds blocks, thereby invalidating existing python objects.
I decided against that for this first version.
So currently when converting a Mesh to a BMesh the bool layers are always created. (and removed if 'all false' when converting back to Mesh). So at the moment they can never be NULL.

So strictly speaking all the asserts and ensure() calls are currently not needed. However, it would be very nice if we could extend the lazy allocation to BMesh, since for a lot of BMesh uses the bool layers aren't needed, therefor I decided to keep the assert/ ensure.

Martijn Versteegh (Baardaap) marked 2 inline comments as done.Jan 5 2023, 3:41 PM
  • Deleting a UV layer on the default cube in edit-mode is crashing for me:

This is fixed.

Still looking for where to limit the user editable name length

source/blender/editors/include/ED_mesh.h
555–556 ↗(On Diff #59067)

For what it's worth, I agree that the abbreviations are unnecessary and make the code harder to read.
In this case it makes it seem like the function does something totally different too-- selecting something instead of just creating the selection attribute.

source/blender/editors/include/ED_uvedit.h
89–90

This struct is very small, it's just four integers-- the same size as Span and other structs we typically pass by value. It makes the code much simpler to pass by value, especially C code without the concept of references. So I'm in favor of passing by value here.

source/blender/editors/mesh/mesh_data.cc
347 ↗(On Diff #59031)

I'm pretty sure it's an edge selection stored on corners because you can select split edges in the UV editor even if they actually come from the same edge on the mesh.

source/blender/python/bmesh/bmesh_py_types_meshdata.c
8

Is it the intention to deprecate BMLoopUV use? (in favor of accessing primitive data).

We didn't talk about that yet. I think it would be good to deprecate BMLoopUV in 4.0 so we can simplify these things in the future.
Eventually we should also change the API so that scripts can add one of the boolean attributes without invalidating all current Python objects referencing BMesh attribute values.
I'm not sure how that would be done yet, but it must be possible.
The last thing we'd need is a more intuitive way to find the name of the boolean sub-attributes. Some Mesh/BMesh helper function would be enough for that IMO.

If UV's should not be accessed via BPy_BMLoopUV, shouldn't Python templates/example code be updated? (or noted as a TODO).

Yes, that might be nice, I think it's fine for that to be a separate patch though.

39–41

Agreed that it's worth keeping the asserts and "ensure" calls in BMesh code to ease the transition to lazily creating the boolean layers in the future.

79

If there is no vertex selection layer, the element is just not selected, it's not an error.
Though this won't be reachable currently because the boolean attributes are always created, it's planned in a next step to create the attributes on-demand.

144

This is fine right now, since the boolean attributes are always created for BMesh. It will have to change in the future when they're created on-demand though.

source/blender/editors/mesh/mesh_data.cc
338 ↗(On Diff #59031)

Actually I can't, since these functions are used in mesh_data.cc so they should be public.

source/blender/editors/mesh/mesh_data.cc
347 ↗(On Diff #59031)

True! These are 'face edges' not 'mesh edges'.

Still wouldn't hurt to document it maybe. I'll have a go a adding a comment.

source/blender/python/bmesh/bmesh_py_types_meshdata.c
8

Currently anything creating new CustomData layers in a BMesh causes all python objects having pointers into that same CustomData set (domain) to have invalid pointers which can possibly crash blender.

That is something that would need to be changed before we could do lazy allocation.

Martijn Versteegh (Baardaap) marked 2 inline comments as done.Jan 5 2023, 7:39 PM
Martijn Versteegh (Baardaap) marked an inline comment as done.
Martijn Versteegh (Baardaap) marked an inline comment as done.Jan 5 2023, 8:18 PM

-Limited the user settable layer name length to 64
-Some small fixes
-Clarifying comments
-updated to latest master

Martijn Versteegh (Baardaap) marked an inline comment as done.Jan 6 2023, 12:38 AM
Martijn Versteegh (Baardaap) added inline comments.
source/blender/makesdna/DNA_customdata_types.h
51

Apparently I had tested it the wrong way. The layername sizes should now be capped at 64 from a user point of view (including when blender appends a sequence numer to make the names unique).

Not the most elegant solution but I couldn't think of a better one.

maybe MAX_CUSTOMDATA_LAYER_NAME_GUI should be MAX_CUSTOMDATA_LAYERNAME_EXTERNAL ?

source/blender/editors/include/ED_mesh.h
555–556 ↗(On Diff #59067)

The new names are confusing but I don't think this relates especially to select/selection terms.
The uv_index is also ambiguous as mentioned in the last review.

Suggest:

  • ED_mesh_uv_map_vert_select_layer_ensure(mesh, uv_layer_index)
  • ED_mesh_uv_map_vert_select_layer_get(mesh, uv_layer_index)
  • ED_mesh_uv_map_vert_pin_layer_ensure(mesh, uv_layer_index)
  • ED_mesh_uv_map_vert_pin_layer_get(mesh, uv_layer_index)

... etc.

source/blender/editors/include/ED_uvedit.h
89–90

I was/am concerned with this adding overhead to draw code. for e.g. this passing 16 byte argument to drawing functions that run on every face & loop (updating edit-mesh draw data), even when the mesh doesn't have UV's (just to check the value is -1 and returning in some cases).

However I couldn't measure a performance difference so I suppose it's OK, it would be good to have a rule of thumb for whats reasonable to pass by value (noticed larger structs being passed by value creeping into our code recently), although some seem like mistakes too.

source/blender/editors/mesh/mesh_data.cc
338 ↗(On Diff #59031)

In this case they should be moved to ED_mesh API (source/blender/editors/mesh/mesh_data.cc seems appropriate).

source/blender/python/bmesh/bmesh_py_types_meshdata.c
8

This is fine but it's information that should be in code-comments, just a brief note on the current state and how it may change in future. Otherwise the reference to legacy #MLoopUV type reads like an outdated comment that should be removed as MLoopUV isn't referenced at all.

39–41

+1, in that case a comment noting these may be lazily allocated in the future would be good to include.

79

That's true for reading the value, for assigning, enabling the selection will be ignored and should raise an exception.

source/blender/editors/mesh/mesh_data.cc
338 ↗(On Diff #59031)

But they are in source/blender/editors/mesh/mesh_data.cc ??

source/blender/editors/mesh/mesh_data.cc
338 ↗(On Diff #59031)

Ah! Was jumping between this an rna_mesh.c & got mixed up, sorry for noise.

Martijn Versteegh (Baardaap) marked 18 inline comments as done.Jan 6 2023, 1:11 PM
Martijn Versteegh (Baardaap) added inline comments.
source/blender/editors/include/ED_mesh.h
555–556 ↗(On Diff #59067)

To me (maybe because english is not my native language) select still looks like a verb in those names and I don't really understand why it would be bad to add the 'ion' to make it absolutely clear. The word order doesn't really help to make the distinction either , as some parts of blender use english grammar order and other parts use 'yoda' order so...

I also have my doubts about uv_layer_index. In the CustomData code the layer_index signifies the index in *all* the layers (of all types) within the domain, while this is not really the layer_index, but the uv_map index (i.e. the n'th uv map). I do agree uv_index is confusing. I propose uv_map_index .

But it's not important enough to me to keep discussing it. At least not within the scope of this patch. That discussion would be better done on the style guide task maybe. And also it's probably more important to be consistent with the surrounding code.

I'll rename them to these new names. using uv_map_index instead of uv_layer_index , let me know if you disagree.

source/blender/editors/include/ED_uvedit.h
89–90

As an aside, some studying of generated code in godbolt made me think passing by reference is not a certified win either.

For small(ish) functions gcc -O3 was inlined when passed by value (struct with 4 integers), but not when passed by reference. Also when passing by reference extra instructions were generated to move the referenced value into registers to work on them, while in the pass-by-value they already were in registers.

I guess it depends on register-pressure which is faster, but imo without profiling it's not really possible to tell which is actually better. So to me passing this by reference/pointer feels like 'premature optimisation' .

Though I must admit I need to suppress an urge to do it anyway, since for me having learned C in the early 90s passing 128 bytes *on the stack* feels like heresy :-D

source/blender/python/bmesh/bmesh_py_types_meshdata.c
39–41

Even when having a BMLoop I still need the offsets...

Martijn Versteegh (Baardaap) marked 2 inline comments as done.Jan 6 2023, 1:17 PM
Martijn Versteegh (Baardaap) marked 5 inline comments as done.Jan 6 2023, 3:40 PM
Martijn Versteegh (Baardaap) added inline comments.
source/blender/python/bmesh/bmesh_py_types_meshdata.c
8

I added a clarifying comment. Also explaining the plan is to add direct access to the bool layers.
I prefer not to make that part of this patch though. I propose to work on that as a separate patch.

73

I added a python exception, but left the assert in for debugmode builds.

Martijn Versteegh (Baardaap) marked an inline comment as done.Jan 6 2023, 3:59 PM
Campbell Barton (campbellbarton) requested changes to this revision.Jan 9 2023, 8:05 AM

Note that all references to "BLI_math_vec_types.hh" need to be changed to "BLI_math_vector_types.hh".

Noticed a strange bug with the patch:

With the default cube, run this in the Python terminal:

C.object.data.uv_layers[0].name = ("X" * 64)

  • Notice the name is: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX (63 characters).

Run the same assignment again:

  • Notice the name is XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.001 (62 characters).

Before this patch the .001 is never added.

source/blender/blenkernel/BKE_attribute.h
134–135 ↗(On Diff #59177)

selection -> select.

source/blender/makesdna/DNA_customdata_types.h
52

Prefer MAX_CUSTOMDATA_LAYER_NAME_NO_EXT over MAX_CUSTOMDATA_LAYER_NAME_GUI which is misleading (the GUI limit also applies to Python API use for example - which isn't related to the GUI).

These defines should include doc-strings noting their purpose and when each should be used.

source/blender/python/bmesh/bmesh_py_types_meshdata.c
8–9

The first line reads if this exposes the MLoopUV struct when it doesn't.

Instead it should be noted that UV's are exposed in a way that is compatible with the legacy MLoopUV data.

75

*spelling* layere -> layer.

This revision now requires changes to proceed.Jan 9 2023, 8:05 AM
source/blender/makesrna/intern/rna_mesh.c
1799

Right, not an issue with this patch but wouldn't hurt to change it here.

source/blender/blenkernel/BKE_customdata.h
29–31

It would be good to have a doc-strings here describing constraints on these names, for e.g. is it essential these never exceed 2 bytes? (MAX_CUSTOMDATA_LAYER_NAME could be referenced too).

source/blender/blenkernel/intern/attribute.cc
248–249 ↗(On Diff #59177)

STRPREFIX macro could be used instead of STREQLEN, avoids assumptions about length of defines.

source/blender/makesdna/DNA_customdata_types.h
52

Correction, as these are prefixes not extensions it could be MAX_CUSTOMDATA_LAYER_NAME_NO_PREFIX.

Martijn Versteegh (Baardaap) marked 8 inline comments as done.Jan 9 2023, 1:17 PM
Martijn Versteegh (Baardaap) added inline comments.
source/blender/python/bmesh/bmesh_py_types_meshdata.c
8–9

Maybe like this?

Martijn Versteegh (Baardaap) marked an inline comment as done.

Handled the inline comments.

-Fixed wrong header

  • fixed typo in comment.
source/blender/python/bmesh/bmesh_py_types_meshdata.c
8–9
76
Martijn Versteegh (Baardaap) marked 2 inline comments as done.Jan 9 2023, 7:20 PM

Some tweaks to comments.

Hans Goudey (HooglyBoogly) set the repository for this revision to rB Blender.

Marking as accepted since I don't think an extra review iteration is needed.

However there is a remaining bug mentioned here: ( https://developer.blender.org/D14365#461708 ) that isn't resolved (where .001 is appended onto the name when it previously didn't).

This patch resolves the issue:

diff --git a/source/blender/blenkernel/intern/attribute.cc b/source/blender/blenkernel/intern/attribute.cc
index 6aae8d1842d..96f6953f678 100644
--- a/source/blender/blenkernel/intern/attribute.cc
+++ b/source/blender/blenkernel/intern/attribute.cc
@@ -153,6 +153,16 @@ static bool bke_id_attribute_rename_if_exists(ID *id,
   return BKE_id_attribute_rename(id, old_name, new_name, reports);
 }
 
+static int bke_id_attribute_name_max_length_calc(const char *name)
+{
+  if ((name[0] == '.') &&
+      (STRPREFIX(name + 1, UV_VERTSEL_NAME ".") || STRPREFIX(name + 1, UV_EDGESEL_NAME ".") ||
+       STRPREFIX(name + 1, UV_PINNED_NAME "."))) {
+    return MAX_CUSTOMDATA_LAYER_NAME;
+  }
+  return MAX_CUSTOMDATA_LAYER_NAME_NO_PREFIX;
+}
+
 bool BKE_id_attribute_rename(ID *id,
                              const char *old_name,
                              const char *new_name,
@@ -167,8 +177,18 @@ bool BKE_id_attribute_rename(ID *id,
     BKE_report(reports, RPT_ERROR, "Attribute name can not be empty");
     return false;
   }
-  if (STREQ(old_name, new_name)) {
-    return false;
+
+  /* NOTE: checking if the new name matches the old name only makes sense when the name
+   * is clamped to it's maximum length, otherwise assigning an over-long name multiple times
+   * will add `.001` suffix unnecessarily. */
+  {
+    const int maxlength = bke_id_attribute_name_max_length_calc(new_name);
+    /* NOTE: An function that performs a clamped comparison without copying would be handy here. */
+    char new_name_clamped[MAX_CUSTOMDATA_LAYER_NAME];
+    BLI_strncpy_utf8(new_name_clamped, new_name, maxlength);
+    if (STREQ(old_name, new_name_clamped)) {
+      return false;
+    }
   }
 
   CustomDataLayer *layer = BKE_id_attribute_search(
@@ -243,12 +263,7 @@ static bool unique_name_cb(void *arg, const char *name)
 bool BKE_id_attribute_calc_unique_name(ID *id, const char *name, char *outname)
 {
   AttrUniqueData data{id};
-  int maxlength = MAX_CUSTOMDATA_LAYER_NAME_NO_PREFIX;
-
-  if (STRPREFIX(name, "." UV_VERTSEL_NAME ".") || STRPREFIX(name, "." UV_EDGESEL_NAME ".") ||
-      STRPREFIX(name, "." UV_PINNED_NAME ".")) {
-    maxlength = MAX_CUSTOMDATA_LAYER_NAME;
-  }
+  const int maxlength = bke_id_attribute_name_max_length_calc(name);
 
   /* Set default name if none specified.
    * NOTE: We only call IFACE_() if needed to avoid locale lookup overhead. */
source/blender/python/bmesh/bmesh_py_types_meshdata.c
9
16
This revision is now accepted and ready to land.Jan 10 2023, 5:16 AM
source/blender/blenkernel/intern/customdata.cc
4544–4550

As this is being done elsewhere it would be better to make into a shared function (my patch added bke_id_attribute_name_max_length_calc), so that could be made public and reused.

source/blender/blenkernel/intern/customdata.cc
4544–4550

I added this to CustomData, since the limitation really comes from there, and I thought it was important to maintain the fact that attributes are implemented with CustomData and not the other way around.