Page MenuHome
Differential D16610

GPU: Compile vulkan shaders to Spir-V binaries.
ClosedPublic
Actions

Authored by Jeroen Bakker (jbakker) on Nov 25 2022, 10:10 AM.

Details

Reviewers
Clément Foucault (fclem)
Miguel Pozo (pragma37)
Maniphest Tasks
T102760: GPU: Implement VKShader.
Commits
rB9c0d822737ff: GPU: Compile vulkan shaders to Spir-V binaries.
Summary

Compile each static shader using shaderc to Spir-V binaries.

The main goal is to make sure that the GLSL created using ShaderCreateInfo and able to compile to Spir-V.
For the second stage a correct pipeline needs to be created and some shader would need more
adjustments (push constants size).

With this patch future changes to GLSL sources can already be checked against vulkan, without the
backend finished.

Mechanism has been tested using MacOS and MoltenVK. For other OS, we should finetune CMake
files to find the right location to shaderc.

************************************************************
*** Build Mon 12 Dec 2022 11:08:07 CET
************************************************************
Shader Test compilation result: 463 / 463 passed (skipped 118 for compatibility reasons)
OpenGL backend shader compilation succeeded.
Shader Test compilation result: 529 / 529 passed (skipped 52 for compatibility reasons)
Vulkan backend shader compilation succeeded.

Diff Detail

Repository
rB Blender
Branch
temp-vulkan-shader
Build Status
Buildable 24916
Build 24916: arc lint + arc unit

Event Timeline

Jeroen Bakker (jbakker) created this revision.Nov 25 2022, 10:10 AM
Harbormaster completed remote builds in B24820: Diff 58011.Nov 25 2022, 10:10 AM
Jeroen Bakker (jbakker) added inline comments.Nov 25 2022, 10:24 AM
source/blender/gpu/CMakeLists.txt
47

Should be removed

282

Should be removed

source/blender/gpu/intern/gpu_context.cc
324

Could be added to master directly.

source/blender/gpu/intern/gpu_init_exit.c
37–38

Should be removed before merging to master.

source/blender/gpu/intern/gpu_shader_builder.cc
106

should be enabled before merging to master.

source/blender/gpu/vulkan/vk_shader.hh
15

Include not needed.

source/blender/gpu/vulkan/vk_shader_log.cc
17

https://github.com/google/shaderc/blob/main/libshaderc_util/src/message_test.cc has test files that parse different source files from the error message.

Perhaps we should add a #pragma to include the correct filename in the error message.

Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 25 2022, 10:25 AM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 25 2022, 10:28 AM
Jeroen Bakker (jbakker) added a project: EEVEE & Viewport.
Jeroen Bakker (jbakker) added a task: T102760: GPU: Implement VKShader..Nov 25 2022, 10:32 AM
Jeroen Bakker (jbakker) added reviewers: Clément Foucault (fclem), Miguel Pozo (pragma37).Nov 25 2022, 10:37 AM
Jeroen Bakker (jbakker) updated this revision to Diff 58016.Nov 25 2022, 2:17 PM
  • Copied over GLSL code gen functions from GLShader.
  • Improve shader log to add correct filenames.
Harbormaster completed remote builds in B24824: Diff 58016.Nov 25 2022, 2:17 PM
Kazashi Yoshioka (AgAmemnno) added a subscriber: Kazashi Yoshioka (AgAmemnno).Nov 25 2022, 9:18 PM
Emi Martinez (EmiMartinez) added a subscriber: Emi Martinez (EmiMartinez).Nov 28 2022, 6:57 PM
Jeroen Bakker (jbakker) updated this revision to Diff 58142.Nov 29 2022, 2:44 PM
  • Remove unused cmake includes/libs.
  • Merge branch 'master' into temp-vulkan-shader
  • Fixed stage interface declaration.
  • regular stage interfaces working.
  • Remove debug code.
Harbormaster completed remote builds in B24853: Diff 58142.Nov 29 2022, 2:44 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 29 2022, 2:47 PM
Jeroen Bakker (jbakker) updated this revision to Diff 58144.Nov 29 2022, 3:16 PM
  • Fixed geometry layout.
Harbormaster completed remote builds in B24854: Diff 58144.Nov 29 2022, 3:16 PM
Jeroen Bakker (jbakker) updated this revision to Diff 58146.Nov 29 2022, 3:55 PM
  • Add glsl shader defines.
  • Add definition of gl_InstanceID.
  • Use correct layout location.
Harbormaster completed remote builds in B24856: Diff 58146.Nov 29 2022, 3:55 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 29 2022, 3:56 PM
Jeroen Bakker (jbakker) updated this revision to Diff 58245.Dec 2 2022, 10:33 AM
  • Fix vulkan compilation of common_smaa_lib.glsl
  • Fix compilation issues in workbench shadow.
  • Fix gpencil shaders.
  • Fix compilation of line dashed shader.
  • Enable OpenGL/Metal shader compilation. (was disabled for debugging).
  • Moved debug code.
Harbormaster completed remote builds in B24909: Diff 58245.Dec 2 2022, 10:33 AM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Dec 2 2022, 10:34 AM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
Jeroen Bakker (jbakker) added inline comments.Dec 2 2022, 11:01 AM
source/blender/gpu/vulkan/vk_shader.cc
340

gpu_InstanceIndex should point to the global instance index, not offset from the base.
Haven't found a solution to provide the correct gpu_BaseInstance (yet).

419

debug code should eventually be removed.

source/blender/gpu/vulkan/vk_shader.hh
15

It is now required.

Jeroen Bakker (jbakker) updated this revision to Diff 58248.Dec 2 2022, 12:05 PM
  • Fix motion patch shaders.
  • Added support for compute and shader storage.
  • Remove warning (pragma once not implemented).
  • Fix sampler is a keyword, do not use a parameter name.
  • Fix Eevee-next depth of field compilation.
  • Fixed lens distortion compilation.
Harbormaster completed remote builds in B24911: Diff 58248.Dec 2 2022, 12:05 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Dec 2 2022, 12:07 PM
Jeroen Bakker (jbakker) updated this revision to Diff 58252.Dec 2 2022, 12:29 PM
  • Revert change to device restriction.
  • Removed debug code.
  • Tweaked gpu_InstanceIndex.
Harbormaster completed remote builds in B24914: Diff 58252.Dec 2 2022, 12:29 PM
Jeroen Bakker (jbakker) requested review of this revision.Dec 2 2022, 12:39 PM
Jeroen Bakker (jbakker) retitled this revision from GPU: Compile vulkan shaders. to GPU: Compile vulkan shaders to Spir-V binaries..
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
Jeroen Bakker (jbakker) updated this revision to Diff 58255.Dec 2 2022, 12:47 PM
  • Removed debug code.
Harbormaster completed remote builds in B24916: Diff 58255.Dec 2 2022, 12:47 PM
Jeroen Bakker (jbakker) updated this revision to Diff 58258.Dec 2 2022, 1:47 PM
  • Remove code that hides missing feature in master as it is now fixed in master.
  • Revert whitespace change.
Harbormaster completed remote builds in B24919: Diff 58258.Dec 2 2022, 1:47 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Dec 2 2022, 1:47 PM
Clément Foucault (fclem) added inline comments.Dec 5 2022, 12:51 AM
source/blender/draw/engines/eevee_next/shaders/eevee_depth_of_field_tiles_flatten_comp.glsl
29–32

The const can be removed in the non vulkan case directly. I don't think it makes a difference.

source/blender/gpu/shaders/gpu_shader_3D_line_dashed_uniform_color_vert.glsl
16–18

I think using choke here is kind of misplaced. It is a violent term for just describing a bug.

18

I'm thinking we could instead remove the macros.

Jeroen Bakker (jbakker) added a comment.EditedDec 5 2022, 4:16 PM

Diff to add shaderc to window. Note that epoch isn't working out of the box so shaderc is still failing.

enable-shaderc.diff1 KBDownload

Fix is first to compile blender to make sure the dll's are copied, after that shaderc seems to work. Will need to find a way to do this as part of shader builder.

Jeroen Bakker (jbakker) updated this revision to Diff 58344.Dec 6 2022, 10:10 AM
Jeroen Bakker (jbakker) marked 3 inline comments as done.
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
  • Remove const keyword from shader global.
  • Converted world_clip_planes macros to functions.
  • Nicer comments describing the issue.
  • Enable performance optimizations.
Harbormaster completed remote builds in B24944: Diff 58344.Dec 6 2022, 10:10 AM
Jeroen Bakker (jbakker) updated this revision to Diff 58346.Dec 6 2022, 10:46 AM
Work around shaderc issue by converting view macros to functions.
Harbormaster completed remote builds in B24946: Diff 58346.Dec 6 2022, 10:46 AM
Jeroen Bakker (jbakker) added inline comments.Dec 6 2022, 10:50 AM
source/blender/draw/intern/shaders/common_view_lib.glsl
251 ↗(On Diff #58346)

Macros using ModelMatrix/ModelMatrixInverse might not be defined yet (or at all) keeping them as macros for now, as it requires bigger changes.
After eevee has been converted to use create infos I don't mind looking into it.

Jeroen Bakker (jbakker) added a parent revision: D16243: EEVEE: Port existing EEVEE shaders and generated materials to use GPUShaderCreateInfo..Dec 6 2022, 10:50 AM
Jeroen Bakker (jbakker) updated this revision to Diff 58503.Dec 12 2022, 11:08 AM
  • Prefix push constants to work around double macro expansion.
  • Vulkan: add cube map array.
Harbormaster completed remote builds in B24999: Diff 58503.Dec 12 2022, 11:08 AM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Dec 12 2022, 11:08 AM
Clément Foucault (fclem) accepted this revision.Dec 12 2022, 11:22 AM

Seems to be in great shape!

source/blender/draw/engines/eevee_next/eevee_defines.hh
12–14

Use #ifndef GPU_SHADER instead.

source/blender/gpu/shaders/gpu_shader_cfg_world_clip_lib.glsl
13 ↗(On Diff #58503)

I would avoid this copy. This might not be optimized by compilers.

This revision is now accepted and ready to land.Dec 12 2022, 11:22 AM
Jeroen Bakker (jbakker) updated this revision to Diff 58505.Dec 12 2022, 12:21 PM
Jeroen Bakker (jbakker) marked 2 inline comments as done.
  • Use GPU_SHADER; not GPU_VULKAN.
  • Don't make a local variable to load the WorldClipPlanes.
Harbormaster completed remote builds in B25000: Diff 58505.Dec 12 2022, 12:21 PM
Closed by commit rB9c0d822737ff: GPU: Compile vulkan shaders to Spir-V binaries. (authored by Jeroen Bakker (jbakker)). · Explain WhyDec 12 2022, 12:25 PM
This revision was automatically updated to reflect the committed changes.
Jeroen Bakker (jbakker) added a commit: rB9c0d822737ff: GPU: Compile vulkan shaders to Spir-V binaries..

Revision Contents
Changeset List

PathSize
build_files/
cmake/
platform/
3 lines
source/
blender/
compositor/
realtime_compositor/
shaders/
4 lines
library/
26 lines
draw/
engines/
eevee_next/
4 lines
shaders/
4 lines
overlay/
shaders/
6 lines
16 lines
workbench/
shaders/
3 lines
intern/
shaders/
4 lines
24 lines
gpu/
4 lines
intern/
6 lines
6 lines
3 lines
12 lines
shaders/
4 lines
vulkan/
16 lines
15 lines
10 lines
4 lines
26 lines
873 lines
16 lines
44 lines

Diff 58255

View Options

build_files/cmake/platform/platform_apple.cmake

Loading...
View Options

source/blender/compositor/realtime_compositor/shaders/compositor_screen_lens_distortion.glsl

Loading...
View Options

source/blender/compositor/realtime_compositor/shaders/library/gpu_shader_compositor_texture_utilities.glsl

Loading...
View Options

source/blender/draw/engines/eevee_next/eevee_defines.hh

Loading...
View Options

source/blender/draw/engines/eevee_next/shaders/eevee_depth_of_field_tiles_flatten_comp.glsl

Loading...
View Options

source/blender/draw/engines/overlay/shaders/overlay_edit_gpencil_guide_vert.glsl

Loading...
View Options

source/blender/draw/engines/overlay/shaders/overlay_motion_path_line_vert.glsl

Loading...
View Options

source/blender/draw/engines/workbench/shaders/workbench_shadow_vert.glsl

Loading...
View Options

source/blender/draw/intern/shaders/common_gpencil_lib.glsl

Loading...
View Options

source/blender/draw/intern/shaders/common_smaa_lib.glsl

Loading...
View Options

source/blender/gpu/CMakeLists.txt

Loading...
View Options

source/blender/gpu/intern/gpu_context.cc

Loading...
View Options

source/blender/gpu/intern/gpu_init_exit.c

Loading...
View Options

source/blender/gpu/intern/gpu_shader.cc

Loading...
View Options

source/blender/gpu/intern/gpu_shader_builder.cc

Loading...
View Options

source/blender/gpu/shaders/gpu_shader_3D_line_dashed_uniform_color_vert.glsl

Loading...
View Options

source/blender/gpu/vulkan/vk_backend.hh

Loading...
View Options

source/blender/gpu/vulkan/vk_backend.cc

Loading...
View Options

source/blender/gpu/vulkan/vk_context.hh

Loading...
View Options

source/blender/gpu/vulkan/vk_context.cc

Loading...
View Options

source/blender/gpu/vulkan/vk_shader.hh

Loading...
View Options

source/blender/gpu/vulkan/vk_shader.cc

Loading...
View Options

source/blender/gpu/vulkan/vk_shader_log.hh

Loading...
View Options

source/blender/gpu/vulkan/vk_shader_log.cc

Loading...