Page MenuHome

Refactor: Move normals out of MVert, lazy calculation
ClosedPublic

Authored by Hans Goudey (HooglyBoogly) on Oct 6 2021, 8:22 PM.

Details

Summary

The Change

As described in T91186, this patch moves mesh vertex normals into a
separate custom data layer, like how face normals are currently stored.
The normal array values are now stored as floats, in a contiguous array.

The patch also changes code to use on-demand calculation for vertex and
face normals. Vertex normals are retrieved with an "ensure" function,
and cached in vertex custom data storage. Face normals are handled in
the same way. The main interface is described in BKE_mesh.c.

Since the logical state of a mesh is now "has normals when necessary",
normals can be retrieved from a const mesh, protected by a mutex.

The goal is to use on-demand calculation for derived data, but leave
room for eager calculation for performance purposes (i.e. modifier
evaluation is threaded, but viewport data generation is not).

Benefits

This moves us closer to a SoA approach rather than the current AoS
paradigm. This makes accessing each value faster, since accessing
a contiguous float3 array can be much more efficient than abstracting
the storage. It also means less memory needs to be loaded to read
mesh positions. And at the cost of more memory usage, normals now
don't have to be converted between float and short all the time.
The best example of this change is in node_geo_input_normal.cc.

In the future, the remaining items can be removed from MVert, and
we will just be left with float3, which has similar benefits (T93602).

This approach makes it much simpler to use lazy calculation of normals,
meaning we can easily defer calculation until it's actually needed.
This is especially important now that we have more opportunities for
temporary meshes in geometry nodes.

Performance

In addition to the theoretical future performance improvements by
making MVert == float3, I've done some basic performance testing
on this patch directly. The performance data I've taken is fairly rough,
but it should be enough to give us an idea about where things
generally stand.

TestBeforeAfterNotes
Mesh Line Primitive36 ms31 ms (1.16x)I'll let this stand in for many areas to show that simply filling in mesh vertex positions is faster, since less memory needs to be accessed. I expect many similar speedups.
Spring Splash Screen1.03-1.06 fps1.06-1.11 fpsIt looks like a slight improvement here. Would need better measurement to have a strong conclusion though.
Sprite Fright Snail Smoosh3.30-3.40 fps3.42-3.50 fpsDue to the non-scientific testing I won't draw a strong conclusion, but this looks like a noticeable improvement.
Set Position with Scaled Normal53 ms39 ms (1.36x)This test shows that using normals in geometry nodes is noticeably faster
Normal Calculation 1.6m Vert Cube25 ms21 ms (1.19x)Calculation of normals itself is noticeably master, possibly because of the lack of float -> short conversion
File Size of 1.6m Vert Cube214.7 MB208.4 MB (1.03x)Files with large meshes can have savings since we no longer save normals unnecessarily

As for memory usage, it may be slightly larger in some cases.
But I didn't notice any difference in the production files I tested.

Tests

Some tests are failing now, for two reasons. First is that normals
are now dirty after the subdivision surface modifier. Even in current
master, the modifier creates different normals than the result
of the Mesh normal calculation. You can observe this by adding a
BKE_mesh_calc_normals call after the modifier in master.

The other test differences are small changes in the result from things
like the displace modifier, where storing normals in floats instead of
shorts gives a slightly different result.

Test results can be updated when the patch is committed.

Future improvements

  • Remove ModifierTypeInfo::dependsOnNormals. Code in each modifier already retrieves normals if they are needed anyway.
  • Copy normals as part of a better CoW system for attributes.
  • Make more areas use lazy instead of eager normal calculation.
  • Remove BKE_mesh_normals_tag_dirty in more places since that is the default state of a new mesh.

Diff Detail

Repository
rB Blender
Branch
temp-vert-normals-cleanup
Build Status
Buildable 19383
Build 19383: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

If reviewers like, I can split some of the changes in mesh_normals.cc into D12402.

Hans Goudey (HooglyBoogly) edited the summary of this revision. (Show Details)
Campbell Barton (campbellbarton) requested changes to this revision.EditedDec 11 2021, 11:54 AM

In general this seems fine.


Regarding saving normals, the changes to CD_MASK_MESH mean normals will be saved to disk, giving a noticeable increase in file size.
(512k subdivided suzanne mesh increased from 67.2mb to 79.3mb).

However these normals aren't added by default (I needed to access the vertex & polygon normals through Python in order to increase the file size)

So I'm wondering why they should be saved at all.

  • If storing the normals on disk is done to avoid recalculation on file load - then it's not happening by default (for static meshes at least),
  • if it's not important, we could avoid the file-size overhead.

It seems reasonable not to save data in the blend file unnecessarily unless there is a compelling reason to do so, since storing additional data adds file-size and run-time memory overhead as well (since this data also gets stored in undo-steps).

source/blender/blenkernel/BKE_mesh.h
407–432

Prefer name BKE_mesh_poly_normals*, as long as we have Mesh.mface and Mesh.totface it's possible to confuse face/poly data structures, also poly is already used for this purpose, eg: BKE_mesh_calc_normals_poly, BKE_mesh_vert_poly_map_create ... etc.

583–598

Wouldn't it make more sense to order this after mverts as done with the function calsl below (currently this is between poly-normals and number of polygon arguments).

These arguments could be moved to taking a single params struct too, but that could be handled later.

source/blender/blenkernel/intern/mesh_normals.cc
142–160

Moving from BLI_task to parallel_for seems out of scope for this patch.

While it's probably fine, it makes any regression here more of a hassle to track down.

You submitted a separate patch that does this which should be bench-marked in isolation to double check there aren't any regressions or undesirable outcomes from the change.

438–446

Note that functions which themselves assert have the down side that the location they report isn't all that helpful.

This could be a macro instead, e.g:

#define BKE_mesh_assert_normals_dirty_or_calculated(mesh) \
  BLI_assert(BKE_mesh_check_normals_dirty_or_calculated(mesh))

Otherwise to find the source of the assertion it's necessary to run re-run inside a debugger, which can be difficult if the file wasn't saved in the state that errors - meaning you loose time trying to redo the error to find out where the assertion took place.

source/blender/modifiers/intern/MOD_displace.c
364

This should be limited to the MOD_DISP_DIR_NOR case.

This revision now requires changes to proceed.Dec 11 2021, 11:54 AM
Hans Goudey (HooglyBoogly) marked 4 inline comments as done.Dec 12 2021, 7:01 PM

Thanks for the review!

I think the reason you weren't seeing the normals saved by default is that they're usually calculated on the evaluated mesh rather than the original.
I agree about saving normals actually. We can always copy normal layers more often later on as an optimization, but for now it's probably simplest not to copy them at all.
A rule "don't save derived data" is probably a good one to follow.

source/blender/blenkernel/BKE_mesh.h
583–598

Good point!

source/blender/blenkernel/intern/mesh_normals.cc
142–160

Yeah, it's probably good to keep this patch simpler! I stripped the changes out of this patch and moved them to D12402

438–446

Hmm, I'm not sure about this honestly. It would require BKE_mesh.h to include DNA_mesh_types.h.
While in practice that probably doesn't change much currently, I think it's reasonable to move in the
direction of having more higher-level APIs like this one, and including the DNA header may work against that.

A backtrace in a debug build isn't perfect, but it often does include enough information to pinpoint a failed assert, even when it's in a separate function.

Given all that, I'd still move it to a macro if you prefer it, just wanted to make my reasoning more clear.

Hans Goudey (HooglyBoogly) marked 2 inline comments as done.
  • Merge branch 'master' into temp-vert-normals-cleanup
  • Fix: Incorrect assert
  • Cleanup: Remove redundant code
  • Fix assert failure from bad threading in ensure functions
  • Change naming BKE_mesh_ensure_face_normals -> BKE_mesh_poly_normals_ensure
  • Displace modifier: Limit normal calculation to the normal displacement case
  • Remove unnecessary changes in mesh_normals.cc
  • Don't write or copy normal layers
  • Merge branch 'master' into temp-vert-normals-cleanup
  • Updates after change in master, cleanup
  • Merge master
  • Update a small bit of new code from the GPU subdivision patch
  • Fix padding in Mesh from changes in master
Jacques Lucke (JacquesLucke) requested changes to this revision.Dec 29 2021, 12:07 PM

I scanned through all the changes. Got a few comments, but overall this patch is looking quite good.
Nice to see that overall the patch actually removes code.

source/blender/blenkernel/intern/mesh.cc
156

Ideally we can share the custom data layer between between multiple meshes, then "copying" the normals would essentially have no cost.

1108

Might be necessary to tag normals dirty when the mask does not include the normal layers.

source/blender/blenkernel/intern/mesh_normals.cc
123

The dirty tag handling of these functions is a bit tricky. Ideally, they should not be necessary, because normals are derived data.
Furthermore, setting removing the dirty tag here is kind of incorrect, because the normals are still invalid after this function ends.

Here is an alternative api that should offer better correctness:

void BKE_mesh_vertex_normals_set(Mesh *mesh, float (*normals)[3]);
void BKE_mesh_poly_normals_set(Mesh *mesh, float (*normals)[3]);

Those functions replace the normal layer. They could also contain an assert that checks that the normals are (almost) the same as the once that would be computed in the canonical way. Removing the dirty tag in these functions would be correct.

Maybe there are places where this api couldn't be used, which are hopefully rare and could potentially be removed over time. Would be good to have a list of such cases to decide how they should be handled. Maybe we need a "legacy api" for this kind of data access.

source/blender/modifiers/intern/MOD_normal_edit.c
337

Why do we have to cast away const here? Seems bad when the normals would be shared between multiple meshes.

This revision now requires changes to proceed.Dec 29 2021, 12:07 PM
Hans Goudey (HooglyBoogly) marked 4 inline comments as done.Dec 29 2021, 9:56 PM
Hans Goudey (HooglyBoogly) added inline comments.
source/blender/blenkernel/intern/mesh.cc
156

Yes, that would be nice. I added a little onto the comment to reflect that.

source/blender/blenkernel/intern/mesh_normals.cc
123

Good point! I spent a while debugging issues that came up when I used these functions in the "ensure" functions, so I agree that clearing the dirty flag here isn't good.

The idea of a "set" API is interesting, but when you want to modify normals already owned by the mesh you need a "get for write" function anyway, and it's a little weird to "set" the normal array to something the mesh already owns.
I also don't really like how that API would require allocating. It's nice to avoid that, since the mesh will end up owning the memory anyway.

I think I found a good compromise though:

  • BKE_mesh_{vertex/poly}_normals_for_write does not clear the dirty flag, but it does make sure the layer isn't referenced elsewhere.
  • BKE_mesh_{vertex/poly}_normals_clear_dirty clears the dirty flag, and we could an assert that the normals are correct in the future too (I think there's too much bad code that messes with the normals to add it right now).
Hans Goudey (HooglyBoogly) marked 2 inline comments as done.
  • Cleanup: Use ensure poly normals in solidify functions
  • Cleanup: Simplify ensure normals functions
  • Fix array modifier build error after recent change in master
  • Improve usage of the "for_write" functions, add a separate function to clear dirty flags
  • Cleanup: Rename BKE_mesh_face_normals_for_write -> BKE_mesh_poly_normals_for_write
  • Cleanup: Remove unused DerivedMesh functions
  • Improve marking normals dirty when copying a mesh
  • Expand on comment about copying layers
  • Merge branch 'master' into temp-vert-normals-cleanup
  • Merge master
  • Small cleanup

Sounds fine from the description.

I've only went in semi-details in the subdiv code. It might be easier for understanding and clarity to remove can_evaluate_normals all together. Will some some confusion which i've mentioned in the inlined comment.
If the work is available in a git branch i can help with doing these changes.

The rest I've only glanced through. Seems rather straightforward changes to described design. Let me know if there is some specific area you want an extra eyes on.

source/blender/blenkernel/intern/subdiv_mesh.c
104–107

Few things here.

Please don't refer to a previous state of code in comments (things like "previously"). Such references are fine for the commit message, but seeing such statement in a comment few years from now is just adding confusion. In this example a better wording would have been:

/* Always initialize the normals buffer since we need a storage for evaluated vertex normal. */

To make things more clear, is it possible to remove eval_final_point_and_vertex_normal and use BKE_subdiv_eval_final_point instead?
After that the remaining step of making things fully nice and tidy would be to remove can_evaluate_normals and rename subdiv_accumulate_vertex_normal_and_displacement to subdiv_accumulate_vertex_displacement

589

No need to mention normals. Those are handled by the generic normals evaluation.

Hans Goudey (HooglyBoogly) marked 2 inline comments as done.

Thanks for the comments.

If the work is available in a git branch i can help with doing these changes.

I've made the changes you suggested, but if you'd like to change something too, the branch is temp-vert-normals-cleanup.

The rest I've only glanced through. Seems rather straightforward changes to described design. Let me know if there is some specific area you want an extra eyes on.

Yes, generally each change is straightforward, especially in isolation. The parts I think are most tenuous are some of the places that use BKE_mesh_vertex_normals_for_write,
but those were already there before, it was all just hidden in MVert.no. The most important parts are probably the API in BKE_mesh.h and the changes at the top of mesh.cc, though I do think Campbell looked at that part.


  • Clarify mesh normal validation
  • Minor formatting change and comment cleanup
  • Remove a couple unnecessary uses of BKE_mesh_calc_normals
  • Fix issue with BKE_mesh_nomain_to_mesh (remesh operator causes failed assert)
  • Merge branch 'master' into temp-vert-normals-cleanup
  • Fix crash restoring undo step mesh to edit mesh
  • Cleanups to subdiv code removing normal calculation based on Sergey's comment
  • Fix warning in solidify non-manifold code
source/blender/blenkernel/intern/subdiv_mesh.c
104–107

Great points, and thanks for the guide of how to clean this up, that was helpful. I think it's in a much better state now.

Testing the patch and noticed there is an issue with particles (see the lower image has kinks in the hair):

Create a hair particle system on the default cube to redo.

Another way to redo the error is running the undo test:

../lib/tests/ui_simulate/run.py --blender='./blender.bin' --tests "test_undo.view3d_mesh_particle_edit_mode_simple" --keep-open

The test passes but you can see the hair looks strange on some halves of the quad where hair is pointing inside the cube.

@Hans Goudey (HooglyBoogly) Thanks for the update. The code indeed is much more clear now. Some minor suggestion: P2702

Testing the patch and noticed there is an issue with particles (see the lower image has kinks in the hair):

Hmm, I can't reproduce this:

I'm also seeing the same results for the quick fur operator as in master. I'm not sure what's happening for you.

@Hans Goudey (HooglyBoogly) Thanks for the update. The code indeed is much more clear now. Some minor suggestion: P2702

Thanks, I added those in this update.

@Hans Goudey (HooglyBoogly) the error is only visible in particle edit-mode (should have mentioned this).

Hans Goudey (HooglyBoogly) edited the summary of this revision. (Show Details)
Hans Goudey (HooglyBoogly) set the repository for this revision to rB Blender.

Thanks for catching the issue in particle edit mode, it was very simple to fix.

  • Fix typo that lead to problems in particle edit mode
  • Apply fixes to subdivision comments from Sergey
  • Use BKE_mesh_vertex_normals_for_write instead of casting away const in two places
  • Merge branch 'master' into temp-vert-normals-cleanup
  • Merge master
  • Fix GPU subdivision crash due to missing vertex normals
Hans Goudey (HooglyBoogly) set the repository for this revision to rB Blender.
  • Merge master and fix build error after math vector types refactor
  • Cleanup: Remove an unnecessary change

LGTM (minor typo noticed inline).


NOTE: smooth meshes load entirely black in 3.0, I'd assume add a version check will have to be added to recalculate normals on load.
source/blender/editors/sculpt_paint/sculpt_intern.h
939

Typo, causes compiler warning.

  • Fix typo
  • Merge master
  • Fix build error in USD code from refactor in master
This revision is now accepted and ready to land.Jan 13 2022, 4:47 PM