With the current approach (having sculpt vertex colors with the new tools only in sculpt mode and loop vertex colors in attribute edit mode with the previous vertex painting tools), both color data modes are still going to be supported. The problem with this approach is:

• Sculpt mode won’t be able to edit loop data as all the Sculpt API functions and iterators are vertex based and changing that will require a huge refactor. It can write the same color data to all the loops of a vertex, but it won’t paint different face corners individually
• We will need to code the vertex - grids conversion functions to keep the layers in sync for vertex and for loops to support multires
• Editing per vertex color data can be supported in the future in attribute edit mode in case more precise editing operations are needed
• Sculpt vertex colors are designed to paint high poly meshes. Adding loop color data support in these tools does not make sense for game engines, but improving attribute paint to edit this data does

In D8218#199660, @Pablo Dobarro (pablodp606) wrote:

With the current approach (having sculpt vertex colors with the new tools only in sculpt mode and loop vertex colors in attribute edit mode with the previous vertex painting tools), both color data modes are still going to be supported. The problem with this approach is:

• Sculpt mode won’t be able to edit loop data as all the Sculpt API functions and iterators are vertex based and changing that will require a huge refactor. It can write the same color data to all the loops of a vertex, but it won’t paint different face corners individually

I don't think the refactor has to be that big. I've looked into it a little bit, to me it seems mostly a matter of extending the vertex iterator to also supporting going over all loops for the paint tools.

The main tricky issue is tools like blur, but I think we can write an accessor for that, to query a vertex color and it will then return the average of all corner colors for that vertex.

• We will need to code the vertex - grids conversion functions to keep the layers in sync for vertex and for loops to support multires

We don't need to support per-corner for multires.

• Editing per vertex color data can be supported in the future in attribute edit mode in case more precise editing operations are needed
• Sculpt vertex colors are designed to paint high poly meshes. Adding loop color data support in these tools does not make sense for game engines, but improving attribute paint to edit this data does

The use case with game engines also includes artistic work. I don't think attribute paint mode is the ideal answer.

I don't think the refactor has to be that big. I've looked into it a little bit, to me it seems mostly a matter of extending the vertex iterator to also supporting going over all loops for the paint tools.

But that won't work for brushes with paint and displacement at the same time. The idea behind this is to be able to modify the displacement and the color in the same loop over the vertices, so performance impact is minimum. Also, there are some other tools (like smear) that rely on having only one color assigned to a position

The use case with game engines also includes artistic work. I don't think attribute paint mode is the ideal answer.

I really don't know any use case where you would need to use these tools for a game engine asset. The whole idea of the sculpt vertex colors is to be able to have a color base in multires while sculpting that you can later bake to a texture to create the final asset, and all these tools require a much higher mesh resolution to work properly than any game engine can handle. The way people use the current MLoopCol data in game engines it to paint low poly models, write ID maps or vector for shaders, none of the new tools in sculpt vertex colors make any sense for that.
The thing is, by doing this we can potentially end up increasing the complexity of the sculpt mode code and spending quite some development time in a use case that is almost never going to be used. To me, it would make more sense to write whatever is necessary of the new sculpt vertex colors features in attribute edit (that way they will also work for face and edge color data) than to add loop iterators support to sculpt mode

To me it still seems a rather arbitrary distinction from a user point of view. You want to paint colors, it's not obvious that it should be done in one mode or the other. We don't have to support it for 2.90, but I wouldn't put this off the table entirely. If at some point the iterators will also be used for texture painting where there also is correspondence to vertices, I'm not sure that loop colors add that much additional complexity.

But can you work on supporting painting of the sculpt vertex colors in vertex paint mode then? To me that's still a requirement for this feature to be considered ready for 2.90.

Or we might have to make this an experimental option, that we only enable by default for 2.91, since we still seems to be quite far from addressing all points in T78041. For example import/export we haven't even look at at all.

I find clear the distinction between per loop and per vertex colors and why you would want to use one or the other, and how tools and UI should be designed accordingly.

• Per loop colors should be used for things that look like Sayonara Wild Hearts. In that case, you are working with low poly meshes and you want precision tools to control the exact color that goes in each face corner (elements selection, element masking, topology gradients...). These are the kind of tools that we are planning to implement in attribute edit mode based on the current mloopcols and they should not be optimized that much for performance. Even if they have a brush tool, it is not necessary for it to have the complexity of a painting brush engine, as it would be something more similar to the circle selection tool than to the sculpt vertex colors paint brush. Making the sculpt vertex color paint brush work in these assets does not make much sense as they don't have the required mesh resolution to capture the effects that brush produces in the surface.

• Per vertex colors should be used for painting assets like the characters and environments from Overwatch or Valorant. They need to be painting in the high resolution version to bake the color base to a texture later. In this case you don't need precise control over the topology elements of the mesh, but you need a brush with proper blending and which produces visible brush strokes in the surface to be able to create textures and gradients. In this case, the brush and tools should also be as fast and memory efficient as possible to be able to work with detailed surfaces of high poly meshes.

This is why I want to make the distinction between editing color data in Attribute edit and Sculpt Vertex Colors as clear as possible as each one should be optimized for its own thing. I don't see the reason to merge these two use cases.

The reason I don't like that idea of using MLoopCol in sculpt mode is that aal tools assume that each element has an editable position, mask value and color, and this will also be true if we implement the PBVH_TEXTURE using the texels as elements. In the case of MLoopCol, each element will have a position, a mask and multiple colors. If we just expose the colors of all loops of that vertex as a single one in the API, we will just wasting memory to save the same color multiple times. If we calculate a new position based on the face corner for each face corner element in order to paint different colors, these positions won't be editable, and the paint and displacement resolution won't match.
Also, It will be confusing that a vertex color layer will work in multires but a Loop Color layer won't. Some other features like the remesher reprojection will also need to be implemented for both loops and vertices.

To me, the feature to consider Sculpt Vertex Colors ready for its use case is the multires support (and maybe some other usability things like rendering the face sets as contour lines to be able to use them). I can try to add per vertex colors support to the current vertex paint mode, but we have to consider that we almost don't have any tool designed for attribute paint yet, so even if we release this functionality in 2.90 it is not going to be that usable. The development of the painting tools for the second use case is way more advanced.

I completely agree on making Sculpt Vertex Colors an experimental feature until the whole workflow is working as it should (which should include features in other areas like brush management UI, multires color and displacement baking, properly finish and testing the new brush engine...), so we can enable everything together in the future and show it as production ready feature. On the other hand, we need to consider that Sculpt Vertex Colors as they are now are fixing a lot of critical issues than the current vertex paint mode has (performance, brush artifacts, color picker behavior, color blending...), so some users will still prefer using them instead of the old ones even if they still have some features missing.

In D8218#199856, @Pablo Dobarro (pablodp606) wrote:

• Per loop colors should be used for things that look like Sayonara Wild Hearts. In that case, you are working with low poly meshes and you want precision tools to control the exact color that goes in each face corner (elements selection, element masking, topology gradients...). These are the kind of tools that we are planning to implement in attribute edit mode based on the current mloopcols and they should not be optimized that much for performance. Even if they have a brush tool, it is not necessary for it to have the complexity of a painting brush engine, as it would be something more similar to the circle selection tool than to the sculpt vertex colors paint brush. Making the sculpt vertex color paint brush work in these assets does not make much sense as they don't have the required mesh resolution to capture the effects that brush produces in the surface.

• Per vertex colors should be used for painting assets like the characters and environments from Overwatch or Valorant. They need to be painting in the high resolution version to bake the color base to a texture later. In this case you don't need precise control over the topology elements of the mesh, but you need a brush with proper blending and which produces visible brush strokes in the surface to be able to create textures and gradients. In this case, the brush and tools should also be as fast and memory efficient as possible to be able to work with detailed surfaces of high poly meshes.

This is why I want to make the distinction between editing color data in Attribute edit and Sculpt Vertex Colors as clear as possible as each one should be optimized for its own thing. I don't see the reason to merge these two use cases.

I understand these two use cases are different, but how can you possibly communicate to the user that for one use case they should use sculpt & paint mode, and for the they you should use attribute mode? A user is not going to guess that. Especially if you e.g. import a model, it will just look like the mesh has vertex colors, and then they have to make a decision about which mode they want to edit the colors depending if they imported per-vertex or per-corner, or do a conversion first?

I think this will all just be very confusing. I see a lot of negative feedback regarding the current state in master already, and I don't see what you are planning to really address that otherwise.

The reason I don't like that idea of using MLoopCol in sculpt mode is that aal tools assume that each element has an editable position, mask value and color, and this will also be true if we implement the PBVH_TEXTURE using the texels as elements. In the case of MLoopCol, each element will have a position, a mask and multiple colors.

Depends what you mean by element. If the element is the loop, it will have one color, and a position and mask shared with other elements.

If we just expose the colors of all loops of that vertex as a single one in the API, we will just wasting memory to save the same color multiple times.

We wouldn't expose it as a single one for writing. Just for the purpose of reading colors for tools like blur and smear, it would be averaged, which is the same type of thing that vertex paint mode does for blurring.

If we calculate a new position based on the face corner for each face corner element in order to paint different colors, these positions won't be editable, and the paint and displacement resolution won't match.

I'm not sure what this refers to. We wouldn't calculate a new position for each corner. If there are tools that work on both positions and colors, we would ensure they edit every face corner and every position exactly once.

Also, It will be confusing that a vertex color layer will work in multires but a Loop Color layer won't.

We can make loop colors work with multires grids as well if we want to, the multires colors are duplicated per corner already. But I don't think it will really be that confusing since per vertex would be the default in Blender, and per corner would be mainly something you create explicitly for low poly models or get when importing them.

Some other features like the remesher reprojection will also need to be implemented for both loops and vertices.

Remeshing should aim to preserve all attributes regardless of which modes support painting the attributes.

To me, the feature to consider Sculpt Vertex Colors ready for its use case is the multires support (and maybe some other usability things like rendering the face sets as contour lines to be able to use them). I can try to add per vertex colors support to the current vertex paint mode, but we have to consider that we almost don't have any tool designed for attribute paint yet, so even if we release this functionality in 2.90 it is not going to be that usable. The development of the painting tools for the second use case is way more advanced.

All we really need right now is the exact same tools that already work in vertex paint mode, but working for any vertex color layer. A redesign of those tools is another matter.

I completely agree on making Sculpt Vertex Colors an experimental feature until the whole workflow is working as it should (which should include features in other areas like brush management UI, multires color and displacement baking, properly finish and testing the new brush engine...), so we can enable everything together in the future and show it as production ready feature. On the other hand, we need to consider that Sculpt Vertex Colors as they are now are fixing a lot of critical issues than the current vertex paint mode has (performance, brush artifacts, color picker behavior, color blending...), so some users will still prefer using them instead of the old ones even if they still have some features missing.

Can you make this into an experimental feature then?

Basically I think we committed this before it was really ready. I was hoping you could address most of the tasks in T78041 by the end of bcon2, but it seems not feasible at this stage.

how can you possibly communicate to the user that for one use case they should use sculpt & paint mode, and for the they you should use attribute mode?

I don't think that is something we should communicate in the UI, that is something users should choose based on the design and functionality of the tools. The same thing happens with the multires/remesher/dyntopo options for sculpting. Of course there are users that are not using the most optimal option for a specific task, but that does not mean we should add UI elements to explain users what the intended sculpting workflow is and how features should be used.

Especially if you e.g. import a model, it will just look like the mesh has vertex colors

Converting between loop, edge and face color attributes to vert colors to be able to use them for painting with the sculpt vertex colors tools is planned in the original design (T71947) and that totally makes sense. Also, all those attributes and domains should be editable from Attribute Edit mode with a simple paint tool. The only limitation I'm proposing is that the advanced paint features and brushes in sculpt mode should only be supported by per vertex colors attributes, and this what should be communicated in the UI.

I think this will all just be very confusing. I see a lot of negative feedback regarding the current state in master already

I would like to know what exactly that negative feedback is, as I'm mostly reading positive feedback about the new functionality and design of the tools, which in my opinion result in a much more functional vertex painting system that what we currently have, even with all its current limitations.

If the element is the loop, it will have one color, and a position and mask shared with other elements.

If the position and mask are shared a we iterate over loops, all loops with the same position and mask will get the same color after using a tool (they will be something similar to multires duplicates). We are going to write the same color info multiple times and we won't get extra color resolution.
What I was proposing is to shift the read position of the loop towards the center of the poly so each loop have a slightly different position compared to the original position of its vertex, this way the painting tools will have color variation between face corners when painting and we will get extra color resolution.

All we really need right now is the exact same tools that already work in vertex paint mode, but working for any vertex color layer

Agree, and I can start working on that (not only for loop and vertex domain, but also for edges and faces).

In conclusion, I agree with unifying the loop and vertex (and even faces and edges) color attributes to be able to edit them in attribute paint with similar tools and color overlays. What I don't agree is adding support to per loop color attributes to sculpt mode, as that will cause even more confusion, it will cost development time, increase the complexity of the advanced painting tools and I still can't see any benefit of doing it for any use case

In D8218#200071, @Pablo Dobarro (pablodp606) wrote:

I don't think that is something we should communicate in the UI, that is something users should choose based on the design and functionality of the tools. The same thing happens with the multires/remesher/dyntopo options for sculpting. Of course there are users that are not using the most optimal option for a specific task, but that does not mean we should add UI elements to explain users what the intended sculpting workflow is and how features should be used.

But if you look at multires/remeshes/dyntopo, you actually edit them all in the same mode with the same tools, regardless of the underlying data structure. That's how vertex colors can work as well.

If the element is the loop, it will have one color, and a position and mask shared with other elements.

If the position and mask are shared a we iterate over loops, all loops with the same position and mask will get the same color after using a tool (they will be something similar to multires duplicates). We are going to write the same color info multiple times and we won't get extra color resolution.

You would not get the same color when using face sets to delimit areas, and this is again exactly the same as it works now in vertex paint mode, where you also need to use selection to do this.

What I was proposing is to shift the read position of the loop towards the center of the poly so each loop have a slightly different position compared to the original position of its vertex, this way the painting tools will have color variation between face corners when painting and we will get extra color resolution.

This is unnecessary, that problem is solved by face sets.

I was thinking on how this can be done and I would rather wait to have Attribute Edit mode implemented before continuing. The thing is, people may say that the current state in master with Sculpt Vertex Colors enabled is confusing because we don't have any functionality or UI of the Attribute Edit project done while the sculpt vertex colors functionality is basically finished.

Adding the loop iterators and face sets to Sculpt Mode to paint loop color layers will work and it is not hard to do (it will take quite some time and it may introduce bugs and limitations in the future), but it really looks like we are just doing this based on the feedback of some users of an unfinished project without being aware of the whole scope of it, instead of thinking if this is really needed for a final use case.

Supporting loop colors in sculpt mode will mean:

• We need to create a PBVH_LOOPS type as the iterators and all API functions need to be aware that the indices refer to loops instead of vertices
• We need a way to manage when the PBVH should be of type FACES or type LOOPS depending on the type of the current vertex color layer. In that case, when sculpting with a loop color layer selected, sculpting will be much slower and take more memory without any specific reason (which in may opinion is more confusing that just showing in the UI that sculpt mode can only paint in vertex colors layers)
• We need to make new functions to copy mesh and color data from loops to multires, and we still don't have a defined way to do this for vertices
• Face sets won't be useful to create paint mask as they are rendered as colors. They should be a different way to render the face sets if we want to use them in Attribute edit mode for masking mesh elements (this also affects sculpt vertex colors in master T77636)

I would rather make some development in the Attribute Edit mode to support painting color in vertex, loops, edges and faces with similar tools to what we have in vertex paint, and them, ask people again if they need to use the tools available in sculpt mode to edit per loop/edge/face color data. Depending on what is needed, we can consider adding support for loop iterators to sculpt mode again.

If you could start developing attribute edit mode that would be great, that kind of thing will also benefit particle nodes.

We need to create a PBVH_LOOPS type as the iterators and all API functions need to be aware that the indices refer to loops instead of vertices
We need a way to manage when the PBVH should be of type FACES or type LOOPS depending on the type of the current vertex color layer. In that case, when sculpting with a loop color layer selected, sculpting will be much slower and take more memory without any specific reason (which in may opinion is more confusing that just showing in the UI that sculpt mode can only paint in vertex colors layers)

I don't think this would require a new PBVH type, just iterators and API functions would need to be updated. Each PBVH node contains faces, and the loops of those faces are then also in that node.

Face sets won't be useful to create paint mask as they are rendered as colors. They should be a different way to render the face sets if we want to use them in Attribute edit mode for masking mesh elements (this also affects sculpt vertex colors in master T77636)

Rendering of colored face sets is indeed an issue, and that needs to be solved somehow for vertex colors in general. But being able to organize the mesh into parts I guess is equally useful for sculpting and editing all kinds of attributes and vertex colors.

I can start adding tools for attribute editing to vertex paint in 2.91 as soon as the attributes panel and UI is in master. After finishing the sculpt vertex colors tasks, there is no new major feature planned for sculpt mode.
Sure, it can be added without adding other PBVH type, but now all code is using the type to know the meaning of the indices and which implementation should be used to get the position, normals and colors.
Also, when building the tree it creates the lists of unique vertices and face vert indices, which also needs to be updated. Rendering code needs to be updated as well.

I think that a much easier solution if we want to support loop colors is creating two functions to get/set the color of an element. In case a loop color layer is selected, just build a vert -> loop map, use the same vertex iterators we currently have and let that function handle the face set per loop masking. To get the color of a vertex given its index, just return the color of the first loop that is unmasked. If this solution is valid, then unifying the colors and supporting both in sculpt mode seems fine.

Also, following the discussion in D8320, sculpt vertex colors are scene linear and mloopcols are srgb, but all of new the painting tools are coded using floats. If we use the set/get functions approach to modify the color data the conversion could be handled there, but I guess this will cause more problems and confusion in the future regarding other features (export/import, baking...)