Change Details

The goal of this tasks is to validate the design of Multires as Blender’s main subdivision based sculpting system. In order to do that, I’m not focusing on the current state of Multires taking into account its current implementation (bugs, missing features, technical debt), but on the design of Multires itself. The idea here is to ask the question “if Multires was working exactly as intended (no bugs, no TODOs, all possible optimizations...), is that what we want for the sculpting/painting modes?” ## Pros - **Tangent space sculpting layers**: This is a feature that even specialized software doesn’t support and which is really important for retouching VFX animation. By having multiple ##CD_MDISP## datalayer, it should be possible to tweak the influence of each one of them on the final displacement. As these layers are in tangent space, this supports any transformation on the base mesh without applying the layers displacement in random directions. - **Supports features that require base + displacement**: Features like displacement eraser or displacement smear rely on having the sculpt geometry stored as a base and a displacement. The current Multires design makes supporting these features straightforward. When Multires is not being used, it is still possible to implement these brushes by creating an operator that generates a base (by smoothing the mesh multiple times) and a displacement. So, this is not a feature that relies on the design of Multires, but it makes it better integrated. - **Sculpting and rendering different levels**: It is possible to use sculpt mode to deform the limit surface and let the modifier evaluate the displacement of other level. With a delete lower operator and unsubdivide, it should be possible to create a temporally base mesh for any level so it can be sculpted as a limit surface. - **Easy to integrate with rigs**: For the rest of the animation system, multires is just like having a vector displacement texture on top of a model. Animating models with sculpted displacement does not require any further workflow or features from other modules. - **Saves a lot of memory**: As it does not store any topology information for the higher levels of the sculpt, it is memory efficient both when sculpt mode is in use and when saving the files. - **Faster PBVH building times**: as the BVH primitives are the grids, building the tree is faster than building it for the individual triangles of a mesh. ## Cons - **No references or indices for edges and loops**: Mesh algorithms that require storing information or topology queries based on edges can’t be implemented. This means like features like fairing, geodesic distances or a better cloth solver can’t be directly available from the ##SculptSession## (I still don’t understand why the current cloth solver works, but its performance is really bad). In order to have those features working, I can only think of workarounds like converting the grids to a mesh or adding some kind of map to store the grids topology as edges and vertex references, which adds extra code that needs to be maintained and misses the point of using grids directly to save memory. - **Computed and displayed resolution does not match**: When working on level 1 of a quad mesh, all vertices will have 4 duplicates which will be deformed independently when computing the brushes and tools. For the user, this means that performance and vertex count won’t match the sculpt mesh that is being rendered. This makes Multires usually slower on level 1 than on level 5. It also creates unexpected results when adding a single subdivision to a mid/high poly mesh, as the vertex count will increase at a rate users won’t expect. - **Allows vertices with invalid data state**: This design allows to have a vertex with multiple mask, colors or visibility values. This can be solved with stitching the grids, but this should be something that the design should not allow. It is the cause of constant bugs in all tools that require storing data per vertex. - **Performance depending on grids dimensions**: In lower levels (on a 2x2 grid), the number of duplicates exposed to the tools is so high that creates performance problems. When the grids are too big, performance decreases because the tool iterations are per grid. This causes that multires optimal performance is only achieved in levels around 5. This is something that users don’t expect. For example, subdividing 4 vert plane 8 times will create a reasonable vertex count for sculpting, but because grids at level 8 are so big, sculpt mode will always update at least a quarter of the plane regardless of the size of the brush. - **PBVH limited to building per grid**: In order to get the PBVH building speed, grids are used as primitives. This means that the number of nodes can’t never be higher than the number of grids. On higher multires levels, this is a problem as the PBVH can’t have as many nodes as it should for optimal performance. - **Undefined behavior with undo and shape keys**: The relation between shape keys in the base mesh, sculpt layers and multiple ##CD_MDISP## layers is not clear. We don´t know if shape keys should also contain the info about the displacement of the grids, if they should only deform the base mesh or both. What undo should do with features like propagation and reshaping from sculpt mode is also not clear. - **Incompatible design with automatic apply base**: With the current Multires design it makes total sense to not apply the deformation to the base mesh automatically. The workflow is more similar to painting a displacement texture than to subdivision based sculpting. As it is exposed in the UI as it was a standard subdivision based sculpting system, this often confuses users. As the main use case for having subdivisions in sculpt mode is to be able to control the surface noise, it does not make much sense that level 0 is not directly available for that. - **Unpredictable displacement interpolation in edit mode**: (To be checked with @joeedh ). It looks like it may not be possible to properly implement grid interpolation in edit mode due to numerical stability problems. - **No support for other custom data layers**: The only datalayer that is supported in Multires is the mask, which is hard coded. Multires does not have any system in place to support editable datalayers in the grids. Mode info in T80609. - **Depends on editable limit surface**: As the high poly version of the sculpt is stored as displacement on top of the limit surface, any other Blender tool that changes the limit surface will break it without any warning to the user (for example, changing the crease values in Edit mode). - **Extremely hard to develop and maintain**: Blender most advanced sculpting tools don’t rely that much on displacing vertices, but more on topology algorithms for calculating transformations (pose, relax, boundary, expand...). Making and debugging these tools to work on grids usually takes 70% of their development time (I measure it when developing the boundary brush). A feature that may require a slightly different topology query (which is often trivial to implement for ##Mesh##) requires an amount of development for Multires that is usually bigger than the tool itself. - **No friendly workflow with other softwares**: Having surface detail stored as displacement in Multires is convenient to work inside Blender, but what other software expect is to handle surface detail at render time with UDIMS and vector displacement textures (Blender supports both). If you have an animated scene with multires, for exporting it to other software you need to bake all the displacement at its highest resolution per frame, which is not ideal. Even if Multires displacement data can be stored as a separate file, it can’t be read by any other render engine. ## My opinion The main technical limitations are about exposing the grids directly to the sculpting code. Because of this, I think that it makes sense for Multires to be used as a different feature. By design, Multires use case is closer to being an alternative to vector displacement for rendering than to a subdivision based sculpting system. To me, it makes sense for Multires to support features like storing displacement into the grids, baking the deformation of modifiers to the grids, reshaping a base mesh based on a displacement, unsubdividing and storing the grids as separate files (it already does all that). What I don’t see any value is in being able to edit the grids displacement directly with the tools. For users, Multires should be exposed in the UI as something similar to baking a vector displacement texture to use in rendering, or “ptex based displacement”. After the fixes and the features added in 2.90, Multires is completely functional for this use case. I would rather prefer starting discussing the design of a dedicated subdivision based sculpting system and leave Multires for what it does well instead of keep trying to add workarounds a hacks to make it meet the requirements of something that was not designed for. We need to consider that supporting the existing and planned features (like sculpt vertex colors or layers) in the current Multires will take a lot of development time that can be used for something else. For Multires, I imagine the workflow to be like: - The user creates a subdivision based sculpt with a dedicated subdivision based sculpting system (not Multires). - When done, the user adds a Multires modifier to the high poly version and uses rebuild subdivisions. This will convert the sculpt to a base mesh + displacement. It is the same concept as baking a displacement texture from the high poly mesh into level 0, but slightly more convenient as it does not require UVs and geometry will look exactly the same as the sculpt. - After having the new low res based mesh, the user can rig and animate the mesh with displacement.

The goal of this tasks is to validate the design of Multires as Blender’s main subdivision based sculpting system. In order to do that, I’m not focusing on the current state of Multires taking into account its current implementation (bugs, missing features, technical debt), but on the design of Multires itself. The idea here is to ask the question “if Multires was working exactly as intended (no bugs, no TODOs, all possible optimizations...), is that what we want for the sculpting/painting modes?” ## Pros - **Tangent space sculpting layers**: This is a feature that even specialized software doesn’t support and which is really important for retouching VFX animation. By having multiple ##CD_MDISP## datalayer, it should be possible to tweak the influence of each one of them on the final displacement. As these layers are in tangent space, this supports any transformation on the base mesh without applying the layers displacement in random directions. - **Supports features that require base + displacement**: Features like displacement eraser or displacement smear rely on having the sculpt geometry stored as a base and a displacement. The current Multires design makes supporting these features straightforward. When Multires is not being used, it is still possible to implement these brushes by creating an operator that generates a base (by smoothing the mesh multiple times) and a displacement. So, this is not a feature that relies on the design of Multires, but it makes it better integrated. - **Sculpting and rendering different levels**: It is possible to use sculpt mode to deform the limit surface and let the modifier evaluate the displacement of other level. With a delete lower operator and unsubdivide, it should be possible to create a temporally base mesh for any level so it can be sculpted as a limit surface. - **Easy to integrate with rigs**: For the rest of the animation system, multires is just like having a vector displacement texture on top of a model. Animating models with sculpted displacement does not require any further workflow or features from other modules. - **Saves a lot of memory**: As it does not store any topology information for the higher levels of the sculpt, it is memory efficient both when sculpt mode is in use and when saving the files. - **Faster PBVH building times**: as the BVH primitives are the grids, building the tree is faster than building it for the individual triangles of a mesh. ## Cons - **No references or indices for edges and loops**: Mesh algorithms that require storing information or topology queries based on edges can’t be implemented. This means like features like fairing, geodesic distances or a better cloth solver can’t be directly available from the ##SculptSession## (I still don’t understand why the current cloth solver works, but its performance is really bad). In order to have those features working, I can only think of workarounds like converting the grids to a mesh or adding some kind of map to store the grids topology as edges and vertex references, which adds extra code that needs to be maintained and misses the point of using grids directly to save memory. - **Computed and displayed resolution does not match**: When working on level 1 of a quad mesh, all vertices will have 4 duplicates which will be deformed independently when computing the brushes and tools. For the user, this means that performance and vertex count won’t match the sculpt mesh that is being rendered. This makes Multires usually slower on level 1 than on level 5. It also creates unexpected results when adding a single subdivision to a mid/high poly mesh, as the vertex count will increase at a rate users won’t expect. - **Allows vertices with invalid data state**: This design allows to have a vertex with multiple mask, colors or visibility values. This can be solved with stitching the grids, but this should be something that the design should not allow. It is the cause of constant bugs in all tools that require storing data per vertex. (see T79317) - **Performance depending on grids dimensions**: In lower levels (on a 2x2 grid), the number of duplicates exposed to the tools is so high that creates performance problems. When the grids are too big, performance decreases because the tool iterations are per grid. This causes that multires optimal performance is only achieved in levels around 5. This is something that users don’t expect. For example, subdividing 4 vert plane 8 times will create a reasonable vertex count for sculpting, but because grids at level 8 are so big, sculpt mode will always update at least a quarter of the plane regardless of the size of the brush. - **PBVH limited to building per grid**: In order to get the PBVH building speed, grids are used as primitives. This means that the number of nodes can’t never be higher than the number of grids. On higher multires levels, this is a problem as the PBVH can’t have as many nodes as it should for optimal performance. - **Undefined behavior with undo and shape keys**: The relation between shape keys in the base mesh, sculpt layers and multiple ##CD_MDISP## layers is not clear. We don´t know if shape keys should also contain the info about the displacement of the grids, if they should only deform the base mesh or both. What undo should do with features like propagation and reshaping from sculpt mode is also not clear. - **Incompatible design with automatic apply base**: With the current Multires design it makes total sense to not apply the deformation to the base mesh automatically. The workflow is more similar to painting a displacement texture than to subdivision based sculpting. As it is exposed in the UI as it was a standard subdivision based sculpting system, this often confuses users. As the main use case for having subdivisions in sculpt mode is to be able to control the surface noise, it does not make much sense that level 0 is not directly available for that. - **Unpredictable displacement interpolation in edit mode**: (To be checked with @joeedh ). It looks like it may not be possible to properly implement grid interpolation in edit mode due to numerical stability problems. - **No support for other custom data layers**: The only datalayer that is supported in Multires is the mask, which is hard coded. Multires does not have any system in place to support editable datalayers in the grids. Mode info in T80609. - **Depends on editable limit surface**: As the high poly version of the sculpt is stored as displacement on top of the limit surface, any other Blender tool that changes the limit surface will break it without any warning to the user (for example, changing the crease values in Edit mode). - **Extremely hard to develop and maintain**: Blender most advanced sculpting tools don’t rely that much on displacing vertices, but more on topology algorithms for calculating transformations (pose, relax, boundary, expand...). Making and debugging these tools to work on grids usually takes 70% of their development time (I measure it when developing the boundary brush). A feature that may require a slightly different topology query (which is often trivial to implement for ##Mesh##) requires an amount of development for Multires that is usually bigger than the tool itself. - **No friendly workflow with other softwares**: Having surface detail stored as displacement in Multires is convenient to work inside Blender, but what other software expect is to handle surface detail at render time with UDIMS and vector displacement textures (Blender supports both). If you have an animated scene with multires, for exporting it to other software you need to bake all the displacement at its highest resolution per frame, which is not ideal. Even if Multires displacement data can be stored as a separate file, it can’t be read by any other render engine. ## My opinion The main technical limitations are about exposing the grids directly to the sculpting code. Because of this, I think that it makes sense for Multires to be used as a different feature. By design, Multires use case is closer to being an alternative to vector displacement for rendering than to a subdivision based sculpting system. To me, it makes sense for Multires to support features like storing displacement into the grids, baking the deformation of modifiers to the grids, reshaping a base mesh based on a displacement, unsubdividing and storing the grids as separate files (it already does all that). What I don’t see any value is in being able to edit the grids displacement directly with the tools. For users, Multires should be exposed in the UI as something similar to baking a vector displacement texture to use in rendering, or “ptex based displacement”. After the fixes and the features added in 2.90, Multires is completely functional for this use case. I would rather prefer starting discussing the design of a dedicated subdivision based sculpting system and leave Multires for what it does well instead of keep trying to add workarounds a hacks to make it meet the requirements of something that was not designed for. We need to consider that supporting the existing and planned features (like sculpt vertex colors or layers) in the current Multires will take a lot of development time that can be used for something else. For Multires, I imagine the workflow to be like: - The user creates a subdivision based sculpt with a dedicated subdivision based sculpting system (not Multires). - When done, the user adds a Multires modifier to the high poly version and uses rebuild subdivisions. This will convert the sculpt to a base mesh + displacement. It is the same concept as baking a displacement texture from the high poly mesh into level 0, but slightly more convenient as it does not require UVs and geometry will look exactly the same as the sculpt. - After having the new low res based mesh, the user can rig and animate the mesh with displacement.

The goal of this tasks is to validate the design of Multires as Blender’s main subdivision based sculpting system. In order to do that, I’m not focusing on the current state of Multires taking into account its current implementation (bugs, missing features, technical debt), but on the design of Multires itself. The idea here is to ask the question “if Multires was working exactly as intended (no bugs, no TODOs, all possible optimizations...), is that what we want for the sculpting/painting modes?” ## Pros - **Tangent space sculpting layers**: This is a feature that even specialized software doesn’t support and which is really important for retouching VFX animation. By having multiple ##CD_MDISP## datalayer, it should be possible to tweak the influence of each one of them on the final displacement. As these layers are in tangent space, this supports any transformation on the base mesh without applying the layers displacement in random directions. - **Supports features that require base + displacement**: Features like displacement eraser or displacement smear rely on having the sculpt geometry stored as a base and a displacement. The current Multires design makes supporting these features straightforward. When Multires is not being used, it is still possible to implement these brushes by creating an operator that generates a base (by smoothing the mesh multiple times) and a displacement. So, this is not a feature that relies on the design of Multires, but it makes it better integrated. - **Sculpting and rendering different levels**: It is possible to use sculpt mode to deform the limit surface and let the modifier evaluate the displacement of other level. With a delete lower operator and unsubdivide, it should be possible to create a temporally base mesh for any level so it can be sculpted as a limit surface. - **Easy to integrate with rigs**: For the rest of the animation system, multires is just like having a vector displacement texture on top of a model. Animating models with sculpted displacement does not require any further workflow or features from other modules. - **Saves a lot of memory**: As it does not store any topology information for the higher levels of the sculpt, it is memory efficient both when sculpt mode is in use and when saving the files. - **Faster PBVH building times**: as the BVH primitives are the grids, building the tree is faster than building it for the individual triangles of a mesh. ## Cons - **No references or indices for edges and loops**: Mesh algorithms that require storing information or topology queries based on edges can’t be implemented. This means like features like fairing, geodesic distances or a better cloth solver can’t be directly available from the ##SculptSession## (I still don’t understand why the current cloth solver works, but its performance is really bad). In order to have those features working, I can only think of workarounds like converting the grids to a mesh or adding some kind of map to store the grids topology as edges and vertex references, which adds extra code that needs to be maintained and misses the point of using grids directly to save memory. - **Computed and displayed resolution does not match**: When working on level 1 of a quad mesh, all vertices will have 4 duplicates which will be deformed independently when computing the brushes and tools. For the user, this means that performance and vertex count won’t match the sculpt mesh that is being rendered. This makes Multires usually slower on level 1 than on level 5. It also creates unexpected results when adding a single subdivision to a mid/high poly mesh, as the vertex count will increase at a rate users won’t expect. - **Allows vertices with invalid data state**: This design allows to have a vertex with multiple mask, colors or visibility values. This can be solved with stitching the grids, but this should be something that the design should not allow. It is the cause of constant bugs in all tools that require storing data per vertex. (see T79317) - **Performance depending on grids dimensions**: In lower levels (on a 2x2 grid), the number of duplicates exposed to the tools is so high that creates performance problems. When the grids are too big, performance decreases because the tool iterations are per grid. This causes that multires optimal performance is only achieved in levels around 5. This is something that users don’t expect. For example, subdividing 4 vert plane 8 times will create a reasonable vertex count for sculpting, but because grids at level 8 are so big, sculpt mode will always update at least a quarter of the plane regardless of the size of the brush. - **PBVH limited to building per grid**: In order to get the PBVH building speed, grids are used as primitives. This means that the number of nodes can’t never be higher than the number of grids. On higher multires levels, this is a problem as the PBVH can’t have as many nodes as it should for optimal performance. - **Undefined behavior with undo and shape keys**: The relation between shape keys in the base mesh, sculpt layers and multiple ##CD_MDISP## layers is not clear. We don´t know if shape keys should also contain the info about the displacement of the grids, if they should only deform the base mesh or both. What undo should do with features like propagation and reshaping from sculpt mode is also not clear. - **Incompatible design with automatic apply base**: With the current Multires design it makes total sense to not apply the deformation to the base mesh automatically. The workflow is more similar to painting a displacement texture than to subdivision based sculpting. As it is exposed in the UI as it was a standard subdivision based sculpting system, this often confuses users. As the main use case for having subdivisions in sculpt mode is to be able to control the surface noise, it does not make much sense that level 0 is not directly available for that. - **Unpredictable displacement interpolation in edit mode**: (To be checked with @joeedh ). It looks like it may not be possible to properly implement grid interpolation in edit mode due to numerical stability problems. - **No support for other custom data layers**: The only datalayer that is supported in Multires is the mask, which is hard coded. Multires does not have any system in place to support editable datalayers in the grids. Mode info in T80609. - **Depends on editable limit surface**: As the high poly version of the sculpt is stored as displacement on top of the limit surface, any other Blender tool that changes the limit surface will break it without any warning to the user (for example, changing the crease values in Edit mode). - **Extremely hard to develop and maintain**: Blender most advanced sculpting tools don’t rely that much on displacing vertices, but more on topology algorithms for calculating transformations (pose, relax, boundary, expand...). Making and debugging these tools to work on grids usually takes 70% of their development time (I measure it when developing the boundary brush). A feature that may require a slightly different topology query (which is often trivial to implement for ##Mesh##) requires an amount of development for Multires that is usually bigger than the tool itself. - **No friendly workflow with other softwares**: Having surface detail stored as displacement in Multires is convenient to work inside Blender, but what other software expect is to handle surface detail at render time with UDIMS and vector displacement textures (Blender supports both). If you have an animated scene with multires, for exporting it to other software you need to bake all the displacement at its highest resolution per frame, which is not ideal. Even if Multires displacement data can be stored as a separate file, it can’t be read by any other render engine. ## My opinion The main technical limitations are about exposing the grids directly to the sculpting code. Because of this, I think that it makes sense for Multires to be used as a different feature. By design, Multires use case is closer to being an alternative to vector displacement for rendering than to a subdivision based sculpting system. To me, it makes sense for Multires to support features like storing displacement into the grids, baking the deformation of modifiers to the grids, reshaping a base mesh based on a displacement, unsubdividing and storing the grids as separate files (it already does all that). What I don’t see any value is in being able to edit the grids displacement directly with the tools. For users, Multires should be exposed in the UI as something similar to baking a vector displacement texture to use in rendering, or “ptex based displacement”. After the fixes and the features added in 2.90, Multires is completely functional for this use case. I would rather prefer starting discussing the design of a dedicated subdivision based sculpting system and leave Multires for what it does well instead of keep trying to add workarounds a hacks to make it meet the requirements of something that was not designed for. We need to consider that supporting the existing and planned features (like sculpt vertex colors or layers) in the current Multires will take a lot of development time that can be used for something else. For Multires, I imagine the workflow to be like: - The user creates a subdivision based sculpt with a dedicated subdivision based sculpting system (not Multires). - When done, the user adds a Multires modifier to the high poly version and uses rebuild subdivisions. This will convert the sculpt to a base mesh + displacement. It is the same concept as baking a displacement texture from the high poly mesh into level 0, but slightly more convenient as it does not require UVs and geometry will look exactly the same as the sculpt. - After having the new low res based mesh, the user can rig and animate the mesh with displacement.