Page MenuHome

Sculpt Attribute API
ClosedPublic

Authored by Joseph Eagar (joeedh) on Jul 20 2022, 10:38 AM.

Details

Summary

This patch adds an attribute API to the sculpt code. I went through many iterations of this in sculpt-dev; this is the result. Note that this patch is related too and based off of D14272.

The idea is to have a unified way of creating temporary or permanent attributes across PBVH_FACES, PBVH_GRIDS and PBVH_BMESH.

Basic Design

The sculpt attribute API can create temporary or permanent attributes (only supported in PBVH_FACES mode). Attributes are created via BKE_sculpt_attribute_ensure.

Attributes can be explicit CustomData attributes or simple array-based pseudo-attributes (this is useful for PBVH_GRIDS and PBVH_BMESH).

SculptAttributePointers

There is a structure in SculptSession for convenience attribute pointers, ss->attrs. Standard attributes should assign these; the attribute API will automatically clear them when the associated attributes are released. For example, the automasking code stores its factor attribute layer in ss->attrs.automasking_factor.

Naming

Temporary attributes should use the SCULPT_ATTRIBUTE_NAME macro for naming, it takes an entry in SculptAttributePointers and builds a layer name.

SculptAttribute

Attributes are referenced by a special SculptAttribute structure, which holds
all the info needed to look up elements of an attribute at run time.

All of these structures live in a preallocated flat array in SculptSession, ss->temp_attributes. This is extremely important. Since any change to the CustomData layout can in principle invalidate every extant SculptAttribute, having them all in one block of memory whose location doesn't change allows us to update them transparently.

This makes for much simpler code and eliminates bugs. To see why this is tricky to get right, imagine we want to create three attributes in PBVH_BMESH mode and we provide our own SculptAttribute structs for the API to fill in. Each new layer will invalidate the CustomData block offsets in the prior one, leading to memory corruption.

Diff Detail

Repository
rB Blender
Branch
temp-sculpt-attribute (branched from master)
Build Status
Buildable 23266
Build 23266: arc lint + arc unit

Event Timeline

Joseph Eagar (joeedh) requested review of this revision.Jul 20 2022, 10:38 AM
Joseph Eagar (joeedh) created this revision.

Removing debug commenting out of CD_FLAG_TEMPORARY setting.

Joseph Eagar (joeedh) edited the summary of this revision. (Show Details)Jul 20 2022, 10:44 AM
Brecht Van Lommel (brecht) requested changes to this revision.Jul 20 2022, 5:41 PM

For review this should be a patch on top of D14272 instead of combining both changes into one, otherwise it's difficult to review.

I didn't look closely at the implementation, but some general notes:

  • I think there should be a clearer distinction with functions for creating/deleting temporary attributes, and generic functions for accessing persistent or temporary attributes.
  • The "release layer" function naming is confusing to me. I would expect that name to go with either reference counting, or some combination of acquire/release functions. From reading the API in BKE_paint.h it's not clear how ownership or lifetimes works. I would clearly reserve "get" functions for purely accessing data, and split or rename BKE_sculptsession_attr_layer_get to indicate it creates temporary attributes.
  • BKE_sculptsession_bmesh_add_layers creates a bunch of temporary layers. Is there a reason this can't go through the same mechanism as other temporary layers?

This kind of abstraction is going to make things slower, where previous a simple array access with an index now becomes a function with various conditionals. Is there a plan to get rid of that kind of overhead eventually, with templates to specialize sculpt tools per mesh data structure?

This revision now requires changes to proceed.Jul 20 2022, 5:41 PM

This kind of abstraction is going to make things slower, where previous a simple array access with an index now becomes a function with various conditionals. Is there a plan to get rid of that kind of overhead eventually, with templates to specialize sculpt tools per mesh data structure?

That's the plan, yes.

Cleanup attribute API:

  • BKE_sculptsession_attr_layer_get is now BKE_sculpt_attribute_ensure
  • New API functions:
    • BKE_sculpt_attribute_get: Get an attribute, returns NULL if it does not exists.
    • BKE_sculpt_attribute_exists: Does an attribute exist.
  • BKE_sculptsession_attr_release_layer is now BKE_sculpt_attribute_destroy
  • BKE_sculptsession_attr_release_all_layers is now BKE_sculpt_attribute_destroy_temporaries
  • BKE_sculptsession_release_stroke_attrs is now BKE_sculpt_destroy_stroke_attrs
  • BKE_sculptsession_update_attr_refs is now BKE_sculpt_attribute_update_refs
Brecht Van Lommel (brecht) requested changes to this revision.Aug 2 2022, 9:12 PM

Can this be rebased now that D14272: Port SculptVertRef refactor to master is master? It's difficult to review a mix of two patches.

source/blender/blenkernel/BKE_paint.h
493

For consistency with function names and comments, perhaps use attribute instead of layer?

So:

struct SculptAttributeParams
struct SculptAttribute
struct SculptAttributePointers
505–518

I think the organization of members in this struct can be better:

/* Domain, data type and name */
eAttrDomain domain;
eCustomDataType proptype;
char name[MAX_CUSTOMDATA_LAYER_NAME];

/* Source layer on mesh/bmesh, if any. */
struct CustomDataLayer *layer;

/* Data stored as flat array. */
void *data;
int elem_size, elem_num;
bool data_for_bmesh; /* Temporary data store as array outside of bmesh. */

/* Data stored per BMesh element. */
int bmesh_cd_offset;

/* Sculpt usage */
SculptLayerParams params;
bool used;
506

int -> eCustomDataType

511

Seems redundant with layer != NULL?

516

I'd rename this to for_bmesh, my initial guess is that this was somehow data stored on bmesh.

518

Add comment explaining what used means in this context.

523–525

There's a convention being made for attribute names in T97452: Attribute identification and semantics. Perhaps use:

.sculpt_persistent_position
.sculpt_persistent_normal
.sculpt_persistent_factor

Or perhaps it should be like this, constructed with the SCULPT_LAYER_NAME macro below?

.sculpt.persistent_position
731

Cusotm -> Custom

732

Rename scl -> attributes for clarity? Or something less obscure at least.

777–778

This could be internal to paint.c instead of in the public API.

If it's public, it's not clear when this kind of updated is expected to be done and should get a more detailed comment.

781–784

These two functions could get more consistent names, perhaps like this:

BKE_sculpt_attributes_destroy_temporary_mode()
BKE_sculpt_attributes_destroy_temporary_stroke()
source/blender/blenkernel/intern/paint.c
1616–1621

It's unclear why this is updating only a subset of SculptAttrPointers. Why this specific subset? Is there a generic function needed to update that struct?

2390

This seems to rely on the assumption that if no PBVH exist, the type can't be a PBVH_GRIDS.

It's not obvious that this is true, or will true after future changes that are unlikely to take this assumption into account.

2400

Can this actually happen, and what are the consequences then?

Or should this have BLI_assert() or BLI_assert_unreachable() and be a fatal error?

2429–2434

Can this workaround be replaced with CustomData_sizeof?

2442–2443

Assigning domain and proptype here is redundant.

2747

Same comment as before, can this go through BKE_sculpt_attribute_ensure and the whole sculpt attribute system, or should this remain an exception?

This revision now requires changes to proceed.Aug 2 2022, 9:12 PM
source/blender/blenkernel/intern/paint.c
2457

In other places where !ss->bm, it says /* BMesh hasn't been created yet */.

BKE_sculpt_attribute_update_refs skips updating any layers with where used == false. Is there a scenario where bmesh gets created later and this layer does not get updated properly?

2547–2557

This could be split out in a separate function used by this and sculpt_attribute_create.

2645–2652

I don't think we should be doing this kind thing manually. We have higher level data structures that can handle dynamically sized lists, like ListBase or blender::Vector.

2751–2752

I would expect these to be defined next to and consistent with SCULPT_PERSISTENT_CO_ID etc.

2859–2861

I think this should be BLI_assert(!scl->used)?

If the layer was already released, the same location in the array may have already been used by another layer. So this should never happen.

2900

This uses printf, others use BLI_assert_unreachable, make it consistent.

2904

Does this work for bmesh? I would expect a call to BM_data_layer_free.

Rebase to latest master

Joseph Eagar (joeedh) marked 17 inline comments as done.

Make requested changes:

  • Renamed layer to attribute in various places
  • dyntopo node attributes now use new API.
  • Made a few functions static in paint.c.
source/blender/blenkernel/intern/paint.c
1616–1621

The idea was that SculptAttrPoints is kept up to date by the code that creates and uses the attributes. Perhaps a single update function is better though.

1616–1621

Hrm. The problem with a single update function is you then have to remember to keep it up to date. Which might be a problem.

2400

I think it does happen for persistent base in multires, which is supposed to still work even if you lose it on exiting sculpt mode.

2457

No. The BMesh is always created first (it actually gets made before the PBVH).

2645–2652

I guess I could use a linked list. It has to be something that won't reallocate the memory, otherwise you end up with lots of subtle bugs.

2747

This is old code, but yes I can make it use the new API.

Brecht Van Lommel (brecht) requested changes to this revision.Aug 22 2022, 8:31 PM

This looks generally fine now, only minor comments.

source/blender/blenkernel/BKE_paint.h
534

It's now used for this, so remove comment?

556–561

These 3 attribute pointers don't seem to be used.

source/blender/blenkernel/intern/paint.c
1616–1621

This still didn't explain why specifically persistent_* attributes need this and not others. Is it because they're the only ones with a lifetime longer than a stroke? If so add a comment explaining that.

2400

If it can happen in practice then it should be handled gracefully instead of BLI_assert_unreachable.

2428

I don't think this is correct for all PBVH types, while BKE_sculptsession_get_totvert gives different results for each. Not sure if it matters in practice, but seems logical to have this consistent.

2645–2652

Ok, if the purpose is to keep the pointers valid on changes I guess it's fine as is.

2652

Make static

2720

Make static.

2724

Should this be const SculptAttributeParams? sculpt_attribute_create seems to modify params while sculptsession_bmesh_add_layers uses the same params variable for two different attributes. I would not expect params to be modified here so would try to at least make it const in the public API.

This revision now requires changes to proceed.Aug 22 2022, 8:31 PM
Joseph Eagar (joeedh) marked 8 inline comments as done.

Make requested patch changes.

source/blender/blenkernel/intern/paint.c
2390

I simplified the code a bit, it still works the same way. I think assuming the null PBVH is not PBVH_GRIDS is fine.

2428

It's correct. ss->totfaces is the same for all modes, even PBVH_GRIDS.

This revision is now accepted and ready to land.Sep 16 2022, 7:05 PM
Joseph Eagar (joeedh) edited the summary of this revision. (Show Details)Sep 16 2022, 9:25 PM
This revision was automatically updated to reflect the committed changes.