Paths

Table of Contentst

EEVEE Viewport: Enabled mipmaps to work properly with cubic interpolation
Needs ReviewPublic
Actions

Authored by Ethan Hall (Ethan1080) on Mar 19 2022, 10:08 PM.

Details

Reviewers

Summary

This patch enables cubic texture filtering to use mipmaps and works with anisotropic filtering.
This means less texture noise with less samples.

Instead of forcing mipmap LoD 0, this approach allows the use of mipmaps.
The main idea is that the distance between the sample points for the four linear texture samples needs to scale with the mipmap that is used.
textureQueryLod, introduced in OpenGL Shading Language Version 4.0, makes this process simpler.

Samples	Original	Patch	Diff x8
1
16
64
16384

Cycles Reference

Comparison vs. Cycles Reference

Samples	Original vs. Cycles Diff x8	Patch vs. Cycles Diff x8
1
16
64
16384

Diff Detail

Repository

rB Blender

Branch

cubic_filtering_mipmaps (branched from master)

Build Status

Buildable 21133
Build 21133: arc lint + arc unit

Event Timeline

Ethan Hall (Ethan1080) requested review of this revision.Mar 19 2022, 10:08 PM

Ethan Hall (Ethan1080) created this revision.

Harbormaster completed remote builds in B21123: Diff 49442.Mar 19 2022, 10:08 PM

Ethan Hall (Ethan1080) edited the summary of this revision. (Show Details)Mar 19 2022, 10:14 PM

Ethan Hall (Ethan1080) added a project: EEVEE & Viewport.

Ethan Hall (Ethan1080) added a subscriber: Clément Foucault (fclem).

Ethan Hall (Ethan1080) edited the summary of this revision. (Show Details)Mar 19 2022, 10:31 PM

Nice patch! However after thinking about it, It feels like it will conflict with anisotropic filtering. The textureGrad GLSL function allows for manual derivatives specification but it seems it is not what you want. The textureGrad works with anisotropy because it defines a rectangular area to sample on the texture itself. In your patch you only only take the maximum value which will sample the lowest resolution LOD from both derivatives resulting in isotropic sampling.

If anything, I would at least use the maximum quality approach and use the min of both derivatives.

However, computing the derivatives manually for the box projection and the cubic interpolation seems quite interesting to have. But for the cubic you will need to find a continuous function to differentiate (it can be non-smooth thought).

source/blender/gpu/shaders/material/gpu_shader_material_tex_image.glsl
141	`uv` is not garantee to be continuous. So you will have cases where you will have gradients discontinuities and possibly a ill defined derivative which will sample the lowest resolution LOD.
289–290	Do not do this. Always use auto formating (`clang-format`) on source code even on GLSL.

This revision now requires changes to proceed.Mar 19 2022, 11:29 PM

Clément Foucault (fclem) added inline comments.Mar 19 2022, 11:37 PM

source/blender/gpu/shaders/material/gpu_shader_material_tex_image.glsl
141	I take back what I've said. The projection is continuous. So this seems correct. For codestyle however, you can remove the `.xy` which is redundant.

Ethan Hall (Ethan1080) planned changes to this revision.Mar 19 2022, 11:37 PM

In D14390#384898, @Clément Foucault (fclem) wrote:

It feels like it will conflict with anisotropic filtering. The textureGrad GLSL function allows for manual derivatives specification but it seems it is not what you want.

Oh, that was it. Anisotropic filtering just does that to the mip levels. I did not realize in all my tests that every time I had filtering enabled anisotropic was also enabled. But that does raise another question, why does turning off the anisotropic setting in preferences not seem to work?

However, computing the derivatives manually for the box projection and the cubic interpolation seems quite interesting to have. But for the cubic you will need to find a continuous function to differentiate (it can be non-smooth thought).

That is the direction I will head with this patch instead.

Ethan Hall (Ethan1080) retitled this revision from EEVEE Viewport: Implemented nicer mip level transitions for image node textures. to (WIP) EEVEE Viewport: Attempting to get mipmaps to work properly with cubic interpolation.Mar 20 2022, 12:50 AM

Ethan Hall (Ethan1080) edited the summary of this revision. (Show Details)

Enabled cubic texture filtering to use mipmaps.

Harbormaster completed remote builds in B21133: Diff 49465.Mar 21 2022, 8:10 AM

Ethan Hall (Ethan1080) retitled this revision from (WIP) EEVEE Viewport: Attempting to get mipmaps to work properly with cubic interpolation to EEVEE Viewport: Enabled mipmaps to work properly with cubic interpolation.Mar 21 2022, 8:32 AM

Ethan Hall (Ethan1080) edited the summary of this revision. (Show Details)

Ethan Hall (Ethan1080) marked 3 inline comments as done.Mar 21 2022, 8:37 AM

Ethan Hall (Ethan1080) edited the summary of this revision. (Show Details)Mar 21 2022, 8:49 AM

Ethan Hall (Ethan1080) added inline comments.Mar 21 2022, 10:22 AM

source/blender/gpu/shaders/material/gpu_shader_material_tex_image.glsl
271	This might be faster.

Ethan Hall (Ethan1080) marked an inline comment as not done.Mar 21 2022, 10:22 AM

Small optimization

Harbormaster completed remote builds in B21137: Diff 49471.Mar 21 2022, 12:02 PM

I think we should follow what cycles does as reference. To me the cubic filter make only sense in a zoomed in a magnifier filter context. So maybe you should try to keep the initial pixel offsets but bias the LOD by 1 because we supply the initial LOD 0 filter with our sampling pattern. Maybe just compute the gradients, divide by 2 the result in both directions (to bias mipmap) and use textureGrad. The bias is only an intuition, it might not actually work.

As for the comparison examples, you should check with a higher frequency texture (like the generated UV textures or a checkerboard) and compare your implementation with the high sample count reference (even maybe against cycles). And try to multiply error by 10 or 100 because it is easy to miss them.

source/blender/gpu/shaders/material/gpu_shader_material_tex_image.glsl
72	So this is where it gets tricky. `textureQueryLod` is part of OpenGL 4.0 but we only guarantee support for 3.3 (for now). This means different hardware would get different results and that's bad.
73	Here it gets tricky too. The fact that you are quantizing the LOD makes it possible to have popping artifacts on LOD transition when the plane is perpendicular to the view.

This revision now requires changes to proceed.Mar 21 2022, 1:41 PM

In D14390#385201, @Clément Foucault (fclem) wrote:

To me the cubic filter make only sense in a zoomed in a magnifier filter context.

That is how OSL's smart filter works. It uses cubic for magnification and bilinear otherwise.
I will create a separate patch to implement smart filtering, for EEVEE. (I am already working on it.)

So maybe you should try to keep the initial pixel offsets but bias the LOD by 1 because we supply the initial LOD 0 filter with our sampling pattern. Maybe just compute the gradients, divide by 2 the result in both directions (to bias mipmap) and use textureGrad. The bias is only an intuition, it might not actually work.
As for the comparison examples, you should check with a higher frequency texture (like the generated UV textures or a checkerboard) and compare your implementation with the high sample count reference (even maybe against cycles). And try to multiply error by 10 or 100 because it is easy to miss them.

I will try some more tests, and determine if biasing the gradients is more true to a reference.

source/blender/gpu/shaders/material/gpu_shader_material_tex_image.glsl
72	Doesn't Blender 3.0+ require OpenGL 4.3 compatibility?
73	I don't think this is actually a problem. I blend the texture size to the next mip's size with the next line of code. `tex_size = 1.0 - 0.5 fract(lod);`

Clément Foucault (fclem) added inline comments.Mar 21 2022, 2:31 PM

source/blender/gpu/shaders/material/gpu_shader_material_tex_image.glsl
72	Nope, we are still kept behind by MacOS implementation until the Metal backend is 100% finished.
73	Ah right I missed that.

Ethan Hall (Ethan1080) marked 3 inline comments as done.Mar 21 2022, 2:47 PM

Ethan Hall (Ethan1080) added inline comments.

source/blender/gpu/shaders/material/gpu_shader_material_tex_image.glsl
72	So this is where it gets tricky. `textureQueryLod` is part of OpenGL 4.0 but we only guarantee support for 3.3 (for now). This means different hardware would get different results and that's bad.
72	I think this Nvidia article agrees broadly with the approach that I took. The implementation is a little bit different though, and it is an older source, so it doesn't make use of a simple `textureQueryLod` equivalent. Here are a few options: I could fall back to not using mipmaps for 3.3, I could wait for Metal to be finished, or I could implement a meta texture lookup to do the work of `textureQueryLod` as described in the Nvidia article.

Mipmapping is a poor choice for normal mapping because it smooths out small bumps when looking from a distance. It would be great to have mipmapping as an option, that you can turn on or off.

Added fallback for OpenGL version less than 400.

Harbormaster completed remote builds in B21212: Diff 49621.Mar 23 2022, 9:03 PM

Ethan Hall (Ethan1080) edited the summary of this revision. (Show Details)Mar 23 2022, 9:23 PM

In D14390#386247, @Mikhail Matrosov (ktdfly) wrote:

Mipmapping is a poor choice for normal mapping because it smooths out small bumps when looking from a distance. It would be great to have mipmapping as an option, that you can turn on or off.

It exists some techniques to bias the roughness from the mipmap details. However, adding that to blender is not really feasible for now.

About Enabling/Disabling mipmaps I can see that working on a per image basis, using a data block option.

All of this is quite unrelated to this Diff and fall into feature request.

In D14390#386247, @Mikhail Matrosov (ktdfly) wrote:

Mipmapping is a poor choice for normal mapping because it smooths out small bumps when looking from a distance. It would be great to have mipmapping as an option, that you can turn on or off.

I agree that mipmapping should be optional. It is listed as a TODO inside the code. I would be willing to help implement the option if @Clément Foucault (fclem) wants the help.

Mipmapping is not always a poor choice for normal maps. (Although, in Blender it currently is.) The real-time graphics approach to the normal mipmaps is to use the loss of high frequency normal information per mip to modify the roughness mipmaps. The loss to roughness conversion can even be calculated for the X and Y directions separately and used to influence anisotropy.

Current WIP:
Implement "smart" interpolation mode for EEVEE.
Implement independent extension settings for X and Y axes.
Implement mirrored repeat wrap mode.

Possible Future Works:
Allow users to provide custom mipmaps for compressed textures (DDS).
Allow users to choose if mipmapping is enabled.
Implement automatic mipmap LoD bias based on render sample count.
Implement an anisotropic shader for EEVEE.
Implement baking mipmaps for compressed textures.
Implement baking normal map mipmap loss to roughness mipmaps/anisotropic.

@Clément Foucault (fclem)
I tested messing with the bias, and it leads to poor results for low sample counts. Not worth it in my opinion.

Ideally, we could bias the LoD based on the sample count automatically for all samplers used by the render.
This way, users that want to use more samples can retain the extra sharpness from sampling lower mips.
And, users that are willing to deal with a little bit of blur can get rid of the nasty aliasing artifacts at low sample counts. This is especially nice when moving the viewport about or viewing animation playback.

Ethan Hall (Ethan1080) marked an inline comment as done.Mar 23 2022, 10:01 PM

In D14390#386279, @Clément Foucault (fclem) wrote:

quite unrelated to this Diff and fall into feature request.

Enabling mipmaps for cubic interpolation without an option to disable it disrupts existing projects that rely on this technique, especially because currently there will be no workaround.

In D14390#386284, @Mikhail Matrosov (ktdfly) wrote:

Enabling mipmaps for cubic interpolation without an option to disable it disrupts existing projects that rely on this technique, especially because currently there will be no workaround.

I see. Then yes, adding mipmapping will need this option.

I would be willing to help implement the option if @Clément Foucault (fclem) wants the help.

You are very welcome to do so.

Mikhail Matrosov (ktdfly) awarded a token.Mar 24 2022, 1:49 AM

Revision Contents
Changeset List

Path

Size

source/

blender/

gpu/

shaders/

material/

gpu_shader_material_tex_image.glsl

37 lines

Diff 49465

View Options

source/blender/gpu/shaders/material/gpu_shader_material_tex_image.glsl

EEVEE Viewport: Enabled mipmaps to work properly with cubic interpolationNeeds ReviewPublicActions