Paths

Table of Contentst

Differential D7060

Workbench Simplification Refactor
ClosedPublic
Actions

Authored by Clément Foucault (fclem) on Mar 8 2020, 2:08 AM.

Details

Reviewers

Jeroen Bakker (jbakker)

Commits

rBc476c36e4008: Workbench Simplification Refactor

Summary

This patch is (almost) a complete rewrite of workbench engine. The features remain unchanged but the code quality is greatly improved.
This also introduce the concept of DRWShaderLibrary to make a simple include system inside the GLSL files.
I did not see any performance change at first glance but this needs more testing.

What it fixes

Infront with transparent object is now possible.
No more trick with the stencil buffer for wireframe masking.
Lower memory usage: Buffers are the same for the transparent and opaque passes. All compositing happens on main framebuffer
Replace WORKBENCH_Material allocations/setup/hash with UBOs containing materials properties in chunks of 4096 materials.
Remove a lot of shader variations.
Unify the opaque and transparent pipelines.
Offscreen rendering broken under some circumstances.

Remaining Regression:

~~Bug with TAA when operators setting overlay values not tagging view_updated. (needs to be fixed ASAP)~~ Done
~~This patch seems to affect roughness a bit. This seems to be caused by the new encoding inside the material.~~ Yes new encoding changes the value as it is rounded before quatization.
~~Hair rendering seems to be a bit brighter. Might be caused by the roughness change.~~ Was intentional.
In front objects don't recieve SSAO (cavity effect) anymore. (design decision)
In front objects don't behave correctly with Depth Of Field effect. (same design decision as SSAO but could be fixed if enough demand for it)
~~All shader variations are compiled when opening blender. (optimization task)~~ This does not seems to be a concern in my tests.

Diff Detail

Repository

rB Blender

Branch

tmp-workbench-rewrite

Build Status

Buildable 7051
Build 7051: arc lint + arc unit

Event Timeline

Clément Foucault (fclem) created this revision.Mar 8 2020, 2:08 AM

Harbormaster completed remote builds in B7015: Diff 22506.Mar 8 2020, 2:08 AM

Clément Foucault (fclem) added a reviewer: Jeroen Bakker (jbakker).Mar 8 2020, 2:09 AM

Clément Foucault (fclem) added a subscriber: Brecht Van Lommel (brecht).

noki paike (amonpaike) awarded a token.Mar 8 2020, 12:44 PM

stefano severi (Tetone) awarded a token.Mar 8 2020, 3:05 PM

Workbench: Fix Ghosting when toggling Xray
Workbench: Optimize out uneeded passes.
Workbench: Fix performance regression caused by TAA samples init

Harbormaster completed remote builds in B7025: Diff 22517.Mar 8 2020, 3:32 PM

Clément Foucault (fclem) edited the summary of this revision. (Show Details)Mar 8 2020, 6:28 PM

Ted Nielsen (brilliant_ape) awarded a token.Mar 9 2020, 1:01 PM

Fran Marz (franMarz) awarded a token.Mar 9 2020, 2:50 PM

I did 50% of the review so far. There are a lot of stuff I like, especially how the materials are now handled. Much better than the previous implementation. I do want to do some more indepth testing by opening old tickets are try to reproduce them. But so far it seems all ok.

source/blender/draw/engines/workbench/shaders/workbench_matcap_lib.glsl
22	This uniform can be moved to the `world_block`. It is constant for the whole frame.
source/blender/draw/engines/workbench/shaders/workbench_transparent_accum_frag.glsl
60	Could be moved in the `#ifdef V3D_LIGHTING_STUDIO` as it is only used there
source/blender/draw/engines/workbench/workbench_data.c
79	`* 8` is not clear to me.
183	use `DRW_state_is_playback` It does the same, but is nowwhere used :-)
source/blender/draw/engines/workbench/workbench_opaque.c
149	Can be removed

From running the workbench tests, the specular seems quite significantly different, especially for hair. Maybe worth looking at before committing?

Put specular boolean inside world_ubo
Remove material data encoding from transparent shaders
Use DRW_state_is_playback instead of copy pasted code
Disable shadowing pass if not necessary
Merge branch 'master' into tmp-workbench-rewrite

Harbormaster completed remote builds in B7051: Diff 22578.Mar 9 2020, 7:28 PM

I found a crash when dealling with lot of instances. I'm going to fix it.
Maybe it's related but AMD pro driver+ linux seems to have issues with instances now.

source/blender/draw/engines/workbench/workbench_data.c
79	2 for material_ubo_data is to avoid an assert. 8 is arbitrary to avoid too much memory fragmentation.

Code-wise I couldn't detect any issues. There are many new stuff in the patch that we also can utilize in other areas! Just some small notes:

The outline is now DPI dependent. We should use a different kernel as it now makes the lines very blurry.
Some aspects of the workbench we should write in a doc as parts are hard to describe in a comment. For example where the roughness/metallic are defined in the shader (OPAQUE_MATERIAL). I could spend some time on it during the code quality day next month

source/blender/draw/engines/workbench/workbench_materials.c
47	We should use `eV3dShadingColorType`. I think I missed this one during the clean up last week
source/blender/draw/engines/workbench/workbench_private.h
444	Use eV3dShadingColorType
source/blender/draw/engines/workbench/workbench_shader.c
120	Add a comment that these needs to be ordered?
source/blender/draw/intern/DRW_render.h
439	com_mask vs compare_mask? At least be consistent.
source/blender/draw/intern/draw_manager_data.c
565	should we remove the parameter?

Fix wrong materialId inside material setup resulting in buffer overflow.

Harbormaster completed remote builds in B7074: Diff 22642.Mar 10 2020, 5:22 PM

Merge branch 'master' into tmp-workbench-rewrite
Address review comments

Harbormaster completed remote builds in B7078: Diff 22648.Mar 10 2020, 7:05 PM

@Brecht Van Lommel (brecht) I did change the hair shading (I just forgot I did). Previously the random shading wasn't applied correctly and made the hair color change too much compared to what it should be.

See this code snippet:

void workbench_hair_random_material(float rand,
                                    inout vec3 color,
                                    inout float roughness,
                                    inout float metallic)
{
  /* Center noise around 0. */
  rand -= 0.5;
  rand *= 0.1;
  /* Add some variation to the hairs to avoid uniform look. */
  metallic = clamp(metallic + rand, 0.0, 1.0);
  roughness = clamp(roughness + rand, 0.0, 1.0);
  /* Modulate by color intensity to reduce very high contrast when color is dark. */
  color = clamp(color + rand * (color + 0.05), 0.0, 1.0);
}

Previous code

#ifdef HAIR_SHADER
  /* Add some variation to the hairs to avoid uniform look. */
  float hair_variation = hair_rand * 0.1;
  color = clamp(color - hair_variation, 0.0, 1.0);
  metallic = clamp(materialColorAndMetal.a - hair_variation, 0.0, 1.0);
  roughness = clamp(materialRoughness - hair_variation, 0.0, 1.0);
#endif

Tests should be updated.

source/blender/draw/intern/draw_manager_data.c
565	This is mostly for API consistency. moreover, it's not excluded we might store the id inside the object in the future.

Jeroen Bakker (jbakker) accepted this revision.Mar 11 2020, 11:39 AM

This revision is now accepted and ready to land.Mar 11 2020, 11:39 AM

Clément Foucault (fclem) edited the summary of this revision. (Show Details)Mar 11 2020, 4:55 PM

Closed by commit rBc476c36e4008: Workbench Simplification Refactor (authored by Clément Foucault (fclem)). · Explain WhyMar 11 2020, 5:14 PM

This revision was automatically updated to reflect the committed changes.

Clément Foucault (fclem) added a commit: rBc476c36e4008: Workbench Simplification Refactor.

Bataev Artem (shader) awarded a token.Mar 11 2020, 8:24 PM

Christopher_Anderssarian mentioned this in T73765: Workbench's X-ray not showing correctly.Mar 16 2020, 4:01 PM

Revision Contents
Changeset List

Path

Size

source/

blender/

blenlib/

intern/

BLI_memblock.c

8 lines

draw/

CMakeLists.txt

47 lines

engines/

overlay/

overlay_engine.c

3 lines

overlay_wireframe.c

10 lines

workbench/

shaders/

workbench_cavity_frag.glsl

workbench_cavity_lib.glsl

97 lines

workbench_common_lib.glsl

154 lines

workbench_composite_frag.glsl

44 lines

workbench_curvature_lib.glsl

47 lines

workbench_data_lib.glsl

34 lines

workbench_deferred_background_frag.glsl

workbench_deferred_composite_frag.glsl

workbench_effect_cavity_frag.glsl

31 lines

workbench_effect_fxaa_frag.glsl

workbench_effect_outline_frag.glsl

24 lines

workbench_effect_smaa_frag.glsl

44 lines

workbench_effect_smaa_vert.glsl

21 lines

workbench_effect_taa_frag.glsl

11 lines

workbench_forward_composite_frag.glsl

workbench_forward_depth_frag.glsl

workbench_forward_transparent_accum_frag.glsl

workbench_ghost_resolve_frag.glsl

workbench_image_lib.glsl

83 lines

workbench_matcap_lib.glsl

30 lines

workbench_material_lib.glsl

21 lines

workbench_merge_infront_frag.glsl

18 lines

workbench_object_outline_lib.glsl

workbench_prepass_frag.glsl

97 lines

workbench_prepass_hair_vert.glsl

94 lines

workbench_prepass_vert.glsl

114 lines

workbench_shader_interface_lib.glsl

21 lines

workbench_shadow_debug_frag.glsl

16 lines

workbench_transparent_accum_frag.glsl

89 lines

workbench_transparent_resolve_frag.glsl

26 lines

workbench_volume_frag.glsl

37 lines

workbench_volume_vert.glsl

3 lines

workbench_world_light_lib.glsl

88 lines

407 lines

workbench_effect_aa.c

workbench_effect_antialiasing.c

421 lines

workbench_effect_cavity.c

182 lines

workbench_effect_dof.c

217 lines

workbench_effect_fxaa.c

workbench_effect_outline.c

55 lines

workbench_effect_taa.c

workbench_engine.h

2 lines

workbench_engine.c

547 lines

workbench_forward.c

workbench_materials.c

526 lines

165 lines

670 lines

95 lines

533 lines

367 lines

workbench_studiolight.c

workbench_transparent.c

180 lines

workbench_volume.c

117 lines

intern/

24 lines

5 lines

29 lines

38 lines

10 lines

draw_manager_shader.c

129 lines

draw_manager_texture.c

7 lines

gpu/

shaders/

gpu_shader_common_obinfos_lib.glsl

11 lines

makesdna/

DNA_view3d_enums.h

4 lines

Commit	Tree	Parents	Author	Summary	Date
49472f4e650d	167b96fc043a	6f3e48b1a6b0 a540d16ee82e	Clément Foucault	Merge branch 'master' into tmp-workbench-rewrite (Show More…)	Mar 9 2020, 7:27 PM
6f3e48b1a6b0	947f787088e5	e61e4dc821c6	Clément Foucault	Disable shadowing pass if not necessary	Mar 9 2020, 7:19 PM
e61e4dc821c6	af558e434541	3eb847a2f651	Clément Foucault	Use DRW_state_is_playback instead of copy pasted code	Mar 9 2020, 7:16 PM
3eb847a2f651	41157faff44c	6bd9a139991a	Clément Foucault	Remove material data encoding from transparent shaders	Mar 9 2020, 7:15 PM
6bd9a139991a	4be354ba3a6c	b944b67e3313	Clément Foucault	Put specular boolean inside world_ubo	Mar 9 2020, 6:54 PM
b944b67e3313	fb53d609db7d	242538eba84a	Clément Foucault	Workbench: Fix performance regression caused by TAA samples init	Mar 8 2020, 2:42 PM
242538eba84a	f6b4bc939489	d87c4e6123be	Clément Foucault	Workbench: Optimize out uneeded passes.	Mar 8 2020, 2:31 PM
d87c4e6123be	4a3cdf9df939	75c87b92774e	Clément Foucault	Workbench: Fix Ghosting when toggling Xray	Mar 8 2020, 2:14 PM
75c87b92774e	d2b459417f91	eff6fa00fd99	Clément Foucault	Workbench Simplification Refactor (Show More…)	Mar 8 2020, 12:59 AM
eff6fa00fd99	734af5611a3d	fa848f602a5a	Clément Foucault	Workbench: Add short explanation about the techniques used for rendering	Mar 7 2020, 8:43 PM
fa848f602a5a	ef7d98629b25	a1fb97fc9452	Clément Foucault	Workbench: Refactor: Fix transparent depth pass optimization logic	Mar 7 2020, 7:50 PM
a1fb97fc9452	b45fb98dd000	e8f236406ce7 6b49a9db5249	Clément Foucault	Merge branch 'master' into tmp-workbench-rewrite (Show More…)	Mar 7 2020, 7:24 PM
e8f236406ce7	2df747269d5d	62ee14b04f6e	Clément Foucault	DRW: Fix offscreen rendering (Show More…)	Mar 7 2020, 7:09 PM
62ee14b04f6e	8c649c78266a	2b38b45887a1	Clément Foucault	Workbench: Refactor: Disable Cavity rendering on infront objects (Show More…)	Mar 7 2020, 7:01 PM
2b38b45887a1	ade58553d433	00ad703bc6c1	Clément Foucault	Cleanup: Workbench: Use _ps suffix for passes	Mar 7 2020, 3:35 AM
00ad703bc6c1	d441944f63f9	58d93a48ea9b	Clément Foucault	Workbench: Refactor: Fix missing global state	Mar 7 2020, 3:19 AM
58d93a48ea9b	464c0667ef15	fccb3c999c9c	Clément Foucault	Workbench: Refactor: Fix background color / alpha in render	Mar 7 2020, 2:59 AM
fccb3c999c9c	39c3e85cc70a	4e9cf5440d0e	Clément Foucault	Workbench: Refactor: Fix Xray transparent depth pass	Mar 7 2020, 2:45 AM
4e9cf5440d0e	ad8b70ece085	8b76070a9f99	Clément Foucault	Workbench: Refactor: Fix memleak in render	Mar 7 2020, 2:27 AM
8b76070a9f99	f8d277b00f5e	a3b52c467545	Clément Foucault	Workbench: Refactor: Draw transparent in wireframe mode.	Mar 7 2020, 2:05 AM
a3b52c467545	5b7d5748e06a	9709c81a5649	Clément Foucault	Workbench: Refactor: Fix ghosting	Mar 7 2020, 1:59 AM
9709c81a5649	0b42e96aba7b	68890f4920fd	Clément Foucault	Workbench: Refactor: Fix texture double free	Mar 7 2020, 1:54 AM
68890f4920fd	75432484e096	7bd778eda5bb	Clément Foucault	Workbench: Refactor: Fix xray toggle not working in rendered mode	Mar 7 2020, 1:20 AM
7bd778eda5bb	748edebde4aa	a9afaaa91b78	Clément Foucault	Workbench: Refactor: Fix render TAA loop	Mar 7 2020, 1:19 AM
a9afaaa91b78	028ef74658d3	29383e130ac3	Clément Foucault	Workbench: Fix issues with cavity and temporal sampling	Mar 7 2020, 12:29 AM
29383e130ac3	a72d9d6b2eca	7a1a6e17fbb1	Clément Foucault	Workbench: Fix heap-buffer-overflow in cavity sample UBO initialization	Mar 6 2020, 11:48 PM
7a1a6e17fbb1	ae939659c537	e76d2e6bd33f	Clément Foucault	Cleanup: Workbench: Codestyle: Rename workbench_aa to workbench_antialiasing	Mar 6 2020, 11:38 PM
e76d2e6bd33f	d72b957f18a0	a713e72952a5	Clément Foucault	Workbench: Refactor: Cleanup	Mar 4 2020, 2:42 AM
a713e72952a5	3f9d6b25e5e1	76fbdd620b82	Clément Foucault	Workbench: Refactor: Remove old unused code	Mar 4 2020, 1:20 AM
76fbdd620b82	1a27ead2bd51	faf0a5e6286a	Clément Foucault	Workbench: Refactor: Fix render pipeline	Mar 3 2020, 11:46 PM
faf0a5e6286a	eac26c4e0d15	a459874e772a	Clément Foucault	Workbench: Refactor: Add Volumetric support	Mar 3 2020, 10:55 PM
a459874e772a	9518ca621bc6	91dfe40ef07a	Clément Foucault	Workbench: Refactor: Antialiasing	Mar 3 2020, 4:45 PM
91dfe40ef07a	f1a7552a79b7	c8407006aa4e	Clément Foucault	Workbench: Refactor: Add Dof Support	Mar 3 2020, 2:49 PM
c8407006aa4e	1efb9bc15932	a102b7ebe5f1	Clément Foucault	Overlay: Fix wireframe after shadow refactor	Mar 3 2020, 2:37 AM
a102b7ebe5f1	03972775c8a0	aae2e1326fe6	Clément Foucault	Workbench: Refactor: Add outline support	Mar 3 2020, 2:29 AM
aae2e1326fe6	8ba85c07010f	606f1d80b39a	Clément Foucault	Workbench: Refactor: Add cavity support	Mar 2 2020, 7:11 PM
606f1d80b39a	b95e99b20079	f426e71e091c	Clément Foucault	Workbench: Refactor: Add Shadow support	Mar 1 2020, 11:50 PM
f426e71e091c	4a19a03ccb15	62a7ac8f17ee	Clément Foucault	Workbench: Refactor: Add hair support + tweak the randomness of hairs color	Mar 1 2020, 9:42 PM
62a7ac8f17ee	72d1883ead8d	73de98e6e091	Clément Foucault	Workbench: Refactor: Add Infront support	Mar 1 2020, 2:26 PM
73de98e6e091	873b8099af9c	d36e63f8db49	Clément Foucault	Workbench: Refactor: Add transparent (xray) support	Mar 1 2020, 2:24 AM
d36e63f8db49	1922d60305bd	e34ce36ea4da	Clément Foucault	Workbench: Refactor: Only allocate id buffer if needed	Feb 29 2020, 11:53 PM
e34ce36ea4da	582c0b53d2b9	76a7ea0b7137	Clément Foucault	Workbench: Refactor: Add specular toggle support	Feb 29 2020, 6:29 PM
76a7ea0b7137	c5888bcac820	46f8367eaca3	Clément Foucault	Workbench: Refactor: Add matcap support	Feb 29 2020, 6:09 PM
46f8367eaca3	5dabb13c82a7	5759e2d6c40d	Clément Foucault	Workbench: Refactor: Add support for Flat shading	Feb 29 2020, 5:22 PM
5759e2d6c40d	cc52c53fc000	e6e740f31766	Clément Foucault	Workbench: Refactor: Add support for texpaint & vertpaint	Feb 29 2020, 4:12 PM
e6e740f31766	7351f61ff049	b6e083358ee4	Clément Foucault	Workbench: Refactor: Support Material data	Feb 28 2020, 5:30 PM
b6e083358ee4	19f628937b57	1e58e68c1a48	Clément Foucault	BLI_memblock: Allow to use BLI_memblock_elem_get using only elem index	Feb 28 2020, 5:29 PM
1e58e68c1a48	f50bfc7e3175	1545c37cdb7e	Clément Foucault	DRW: Manager: Expose resource handle getter	Feb 28 2020, 5:28 PM
1545c37cdb7e	16ee034ea317	616545b5cfaa	Clément Foucault	Workbench: Refactor: Code reorganization	Feb 27 2020, 5:58 PM
616545b5cfaa	1eb9689333d6	4c4e798f3529	Clément Foucault	DRW: Shader: Add shader library manager (Show More…)	Feb 27 2020, 3:06 PM

Workbench Simplification RefactorClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 22578

source/blender/blenlib/intern/BLI_memblock.c

source/blender/draw/CMakeLists.txt

source/blender/draw/engines/overlay/overlay_engine.c

source/blender/draw/engines/overlay/overlay_wireframe.c

source/blender/draw/engines/workbench/shaders/workbench_cavity_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_cavity_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_common_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_composite_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_curvature_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_data_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_deferred_background_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_deferred_composite_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_effect_cavity_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_effect_fxaa_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_effect_outline_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_effect_smaa_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_effect_smaa_vert.glsl

source/blender/draw/engines/workbench/shaders/workbench_effect_taa_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_forward_composite_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_forward_depth_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_forward_transparent_accum_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_ghost_resolve_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_image_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_matcap_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_material_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_merge_infront_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_object_outline_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_prepass_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_prepass_hair_vert.glsl

source/blender/draw/engines/workbench/shaders/workbench_prepass_vert.glsl

source/blender/draw/engines/workbench/shaders/workbench_shader_interface_lib.glsl

source/blender/draw/engines/workbench/shaders/workbench_shadow_debug_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_transparent_accum_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_transparent_resolve_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_volume_frag.glsl

source/blender/draw/engines/workbench/shaders/workbench_volume_vert.glsl

source/blender/draw/engines/workbench/shaders/workbench_world_light_lib.glsl

source/blender/draw/engines/workbench/solid_mode.c

source/blender/draw/engines/workbench/transparent_mode.c

source/blender/draw/engines/workbench/workbench_data.c

source/blender/draw/engines/workbench/workbench_deferred.c

source/blender/draw/engines/workbench/workbench_effect_aa.c

source/blender/draw/engines/workbench/workbench_effect_antialiasing.c

source/blender/draw/engines/workbench/workbench_effect_cavity.c

source/blender/draw/engines/workbench/workbench_effect_dof.c

source/blender/draw/engines/workbench/workbench_effect_fxaa.c

source/blender/draw/engines/workbench/workbench_effect_outline.c

source/blender/draw/engines/workbench/workbench_effect_taa.c

source/blender/draw/engines/workbench/workbench_engine.h

source/blender/draw/engines/workbench/workbench_engine.c

source/blender/draw/engines/workbench/workbench_forward.c

source/blender/draw/engines/workbench/workbench_materials.c

source/blender/draw/engines/workbench/workbench_opaque.c

source/blender/draw/engines/workbench/workbench_private.h

source/blender/draw/engines/workbench/workbench_render.c

source/blender/draw/engines/workbench/workbench_shader.c

source/blender/draw/engines/workbench/workbench_shadow.c

source/blender/draw/engines/workbench/workbench_studiolight.c

source/blender/draw/engines/workbench/workbench_transparent.c

source/blender/draw/engines/workbench/workbench_volume.c

source/blender/draw/intern/DRW_render.h

source/blender/draw/intern/draw_common.h

source/blender/draw/intern/draw_hair.c

source/blender/draw/intern/draw_manager.c

source/blender/draw/intern/draw_manager_data.c

source/blender/draw/intern/draw_manager_shader.c

source/blender/draw/intern/draw_manager_texture.c

source/blender/gpu/shaders/gpu_shader_common_obinfos_lib.glsl

source/blender/makesdna/DNA_view3d_enums.h

Workbench Simplification Refactor
ClosedPublic
Actions

Revision Contents
Changeset List