Page MenuHome

EEVEE: Depth of field: New implementation
ClosedPublic

Authored by Clément Foucault (fclem) on Jan 28 2021, 8:57 PM.
Tags
None
Tokens
"Love" token, awarded by ktdfly."Love" token, awarded by billreynish."Burninate" token, awarded by tako."Love" token, awarded by zanqdo."Love" token, awarded by GeorgiaPacific."Love" token, awarded by Shimoon."Love" token, awarded by vr_sebas."Love" token, awarded by Dabi."Love" token, awarded by TimBrown."Love" token, awarded by 123ertyy."Love" token, awarded by nacioss."Love" token, awarded by rbx775."Like" token, awarded by gritche."Like" token, awarded by duarteframos."Love" token, awarded by activemotionpictures."Burninate" token, awarded by ankitm."Love" token, awarded by Leroy."Love" token, awarded by abdoubouam."Love" token, awarded by Andrea_Monzini."Love" token, awarded by tilapiatsu."Love" token, awarded by victoriumist."100" token, awarded by winnertakesteve."Love" token, awarded by Isfuelo."Burninate" token, awarded by shader."Love" token, awarded by DiogoX2."Love" token, awarded by DimitriBastos."Love" token, awarded by kst."Love" token, awarded by wilBr."Burninate" token, awarded by spiegelball."Love" token, awarded by Schiette."Love" token, awarded by Alumx."Love" token, awarded by PatrickWilson."Love" token, awarded by irfan."100" token, awarded by franMarz."Mountain of Wealth" token, awarded by jbakker."Love" token, awarded by beyond."Love" token, awarded by julperado."Burninate" token, awarded by Draise."Love" token, awarded by 0o00o0oo."Burninate" token, awarded by RedMser."100" token, awarded by gilberto_rodrigues."Love" token, awarded by wo262."Love" token, awarded by YAFU."Love" token, awarded by Schamph.

Details

Summary

This is a complete refactor over the old system. The goal was to increase quality
first and then have something more flexible and optimised.

This fixes issues we had with the old system which were:

  • Too much overdraw (low performance).
  • Not enough precision in render targets (hugly color banding/drifting).
  • Poor resolution near in-focus regions.
  • Wrong support of orthographic views.
  • Missing alpha support in viewport.
  • Missing bokeh shape inversion on foreground field.
  • Issues on some GPUs. (see T72489) (But I'm sure this one will have other issues as well heh...)
  • Fix T81092

I chose Unreal's Diaphragm DOF as a reference / goal implementation.
It is well described in the presentation "A Life of a Bokeh" by Guillaume Abadie.
You can check about it here https://epicgames.ent.box.com/s/s86j70iamxvsuu6j35pilypficznec04

Along side the main implementation we provide a way to increase the quality by jittering the
camera position for each sample (the ones specified under the Sampling tab).

The jittering is dividing the actual post processing dof radius so that it fills the undersampling.
The user can still add more overblur to have a noiseless image, but reducing bokeh shape sharpness.

Effect of overblur (left without, right with):

The actual implementation differs a bit:

  • Foreground gather implementation uses the same "ring binning" accumulator as background but uses a custom occlusion method. This gives the problem of inflating the foreground elements when they are over background or in-focus regions. This is was a hard decision but this was preferable to the other method that was giving poor opacity masks for foreground and had other more noticeable issues. Do note it is possible to improve this part in the future if a better alternative is found.
  • Use occlusion texture for foreground. Presentation says it wasn't really needed for them.
  • The TAA stabilisation pass is replace by a simple neighborhood clamping at the reduce copy stage for simplicity.
  • We don't do a brute-force in-focus separate gather pass. Instead we just do the brute force pass during resolve. Using the separate pass could be a future optimization if needed but might give less precise results.
  • We don't use compute shaders at all so shader branching might not be optimal. But performance is still way better than our previous implementation.
  • We mainly rely on density change to fix all undersampling issues even for foreground (which is something the reference implementation is not doing strangely).

Remaining issues (not considered blocking for me):

  • Slight defocus stability: Due to slight defocus bruteforce gather using the bare scene color, highlights are dilated and make convergence quite slow or imposible when using jittered DOF (or gives )
  • Slight defocus inflating: There seems to be a 1px inflation discontinuity of the slight focus convolution compared to the half resolution. This is not really noticeable if using jittered camera. Fixed
  • Foreground occlusion approximation is a bit glitchy and gives incorrect result if the a defocus foreground element overlaps a farther foreground element. Note that this is easily mitigated using the jittered camera position.
  • Foreground is inflating, not revealing background. However this avoids some other bugs too as discussed previously. Also mitigated with jittered camera position.
  • Sensor vertical fit is still broken (does not match cycles).
  • Scattred bokeh shapes can be a bit strange at polygon vertices. This is due to the distance field stored in the Bokeh LUT which is not rounded at the edges. This is barely noticeable if the shape does not rotate.
  • Sampling pattern of the jittered camera position is suboptimal. Could try something like hammersley or poisson disc distribution.Used hexaweb sampling pattern which is not random but has better

stability and overall coverage.

  • Very large bokeh (> 300 px) can exhibit undersampling artifact in gather pass and quite a bit of bleeding. But at this size it is preferable to use jittered camera position.

Codewise the changes are pretty much self contained and each pass are well documented.
However the whole pipeline is quite complex to understand from bird's-eye view.

Notes:

  • There is the possibility of using arbitrary bokeh texture with this implementation. However implementation is a bit involved.
  • Gathering max sample count is hardcoded to avoid to deal with shader variations. The actual max sample count is already quite high but samples are not evenly distributed due to the ring binning method.
  • While this implementation does not need 32bit/channel textures to render correctly it does use many other textures so actual VRAM usage is higher than previous method for viewport but less for render. Textures are reused to avoid many allocations.
  • Bokeh LUT computation is fast and done for each redraw because it can be animated. Also the texture can be shared with other viewport with different camera settings.

Diff Detail

Repository
rB Blender
Branch
eevee-dof-refactor
Build Status
Buildable 12489
Build 12489: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Super excited for this!

One of the issues from the old system being fixed is listed as "Missing alpha support in viewport."

Is this what I think it means? If we have planes with textures on them with alpha information, now DoF will be correctly supported (as in T62738 issue will be finally fixed)?

Super excited for this!

One of the issues from the old system being fixed is listed as "Missing alpha support in viewport."

Is this what I think it means? If we have planes with textures on them with alpha information, now DoF will be correctly supported (as in T62738 issue will be finally fixed)?

No this isn't about this. It's about background being black when putting world opacity to 0.0 or using holdout materials.

This is a major highlight feature of 2.93. Let's get the nightly and begin testing. :-)

Thank you Clément Foucault for your technical expertise and artistic passion. It is rare to possess that synergy.

Clément Foucault (fclem) retitled this revision from EEVEE: Depth of field: Improve scatter pass to EEVEE: Depth of field: New implementation.Jan 29 2021, 3:28 PM
  • EEVEE: Depth of field: Fix tile prediction and fast gather threshold
  • EEVEE: Depth of field: Bleed more foreground over in-focus area
  • EEVEE: Depth of field: Improve slight defocus opacity estimation
  • EEVEE: Depth of field: Fix incorrect prediction for holefill tiles

I've just downloaded the nightly master branch blender-2.93.0-016bd284fbe6-linux64.
I want to know if these updates are going to solve problems when using texture alpha planes getting corrected de-focused especially when the focus position is falling behind the alpha plane.
Switching the material to Alpha Hashed not solves the issue completely because the edges of the alpha channel are not correct blurred then.

I've attached a sample case where the issue is being experienced.
Thanks Clement for the hard-work as always.

I want to know if these updates are going to solve problems when using texture alpha planes getting corrected de-focused especially when the focus position is falling behind the alpha plane.
Switching the material to Alpha Hashed not solves the issue completely because the edges of the alpha channel are not correct blurred then.

Yes it is not correctly blurred because of the listed limitations. Use jittered camera position to fix the issue.
Alpha blended materials will not be compatible with the PostProcess method. If you want accurate blur of alpha blended material, set max sprite size to 0 and enable jitter camera. You will need a very high sample count but there is no other workaround (other than setting your material to alpha hashed).

I dont see much of an difference to the old one

I did a first pass. Still need to go over the shaders and need to do a stress-test. So far only minor stuff.

source/blender/blenloader/intern/versioning_280.c
5126

Use DNA_struct_elem_find to detect if the file needs to be migrated.

source/blender/draw/engines/eevee/eevee_depth_of_field.c
24

Typo: There are 2 methods

285

Is there a reason why this isn't EEVEE_depth_of_field_init?

source/blender/draw/engines/eevee/eevee_lookdev.c
154

Seems like an unrelated change and can be committed directly to master.

source/blender/draw/engines/eevee/eevee_shaders.c
1079

Use bool to keep interface clean. Cast to int inside the function. This comment is also for the rest of the functions in this file.

source/blender/gpu/opengl/gl_framebuffer.cc
145

Seems like an unrelated change can be applied in master.

source/blender/makesdna/DNA_scene_types.h
485

unrelated change. Check clang-format settings. I saw several of them. Also in Eevee codebase.

source/blender/blenloader/intern/versioning_280.c
5126

Shouldn't this be added to versioning_290.c

source/blender/draw/engines/eevee/eevee_depth_of_field.c
285

Could be related to Bokeh LUT computation is fast and done for each redraw because it can be animated

Dabi awarded a token.Feb 1 2021, 5:47 PM

What a hell of an engineering! Code and algorithm is fine. I do have some remarks on naming (foreground vs background) and some related effects I see in the bokeh shape.

  • Foreground bokeh (camera jitter off) is upside down compared to cycles. When camera jitter is on they match.
  • Background bokeh does not keep its shape (camera jitter off). Might be by design due to the downsampling and reduction. Renderdoc shows it in the foreground passes.
  • There seems to be a naming issue what is foreground and what is background. I expected that everything further away from the CoC would be background and everything nearer would be foreground.
  • Might be related rotation of the bokeh is also reversed compared to cycles.

Something to keep in mind is that combining camera jittering and screen spaced DoF leads to some inconsistencies. The bokeh shape could be applied twice (one from the jittering and one when actually doing the Screen space DoF. Would it be an idea to always use a circular bokeh when camera jittering is enabled?

Clément Foucault (fclem) marked 8 inline comments as done.
  • EEVEE: Depth of field: Fix bokeh shape not inverted for scatter
  • EEVEE: Depth of field: Fix bokeh shape rotation inverted
  • EEVEE: Depth of field: Add slight defocus bokeh shape foreground flip
  • EEVEE: Depth of field: Move versioning code to versioning_290.c
  • EEVEE: Depth of field: Fix typo and wrong ownership
  • EEVEE: Depth of field: Remove studiolight related changes
  • EEVEE: Depth of field: Cleanup eevee_shaders interface
  • Merge branch 'master' into eevee-dof-refactor
  • Cleanup: Fix formating
Clément Foucault (fclem) marked an inline comment as done.Feb 3 2021, 6:02 PM

About the jittered DoF using the the same shape for the FX dof: This is to avoid rounding the shape. The shape is going to be smoothed no matter what kernel we chose (because of the non-separable nature of the filter). Using the same kernel only expands the shape a bit but will not round the shape.

source/blender/draw/engines/eevee/eevee_lookdev.c
154

This was used previously to check if DoF needed an alpha channel. But I realized rapidly that R11G11B10 format had too little precision for large scattered bokeh. I will remove this.

Yes, now the bokeh image matches the jittering it makes more sense and would lead to sharper edges.
I don't see anything holding this back.
Great work!

source/blender/makesrna/intern/rna_scene.c
7372

typo: neighborhood

7398

typo: blurring

This revision is now accepted and ready to land.Feb 5 2021, 10:53 AM
  • EEVEE: Depth of Field: Improves denoise filter
  • EEVEE: Depth of Field: Improves jittered camera sampling pattern

I just tried this out, is this the intended behavior?


I used a point light with a smaller size to test the bokeh, it seems it's been turned into an LED lamp or something, it looks strange

I am happy that it fixed the Alpha Blend problem, the original one would have blurred the entire badge, but the original bokeh looks more like it:

If I leave a part of the original on:


It would still be this strange LED effect, just with larger light bulbs, and the entire badge is blurred by the original system.

Is this the intended behavior?
This is the file

I just tried this out, is this the intended behavior?

yes this is intented behavior. unfortunately for cases like that you just have to bump the sample count very high.