Page MenuHome

GHOST: Vulkan Backend.
ClosedPublic

Authored by Jeroen Bakker (jbakker) on Nov 9 2021, 12:34 PM.

Details

Summary

This adds a vulkan backend to GHOST. The code was extracted from the
tmp-vulkan branch. The main difference with the original code is that
GHOST isn't responsible for fallback. For Metal backend there is already
an idea that the GPU module is responsible for the fallback, not the system.

For Blender we target Vulkan 1.2 at the time of this patch.
MoltenVK (needed to convert Vulkan calls to Metal) has been added as
a separate package.

This patch isn't useful for end-users, currently when starting blender with
--gpu-backend vulkan it would crash as the VBBackend doesn't initialize
the expected global structs in the GPU module.

Validated to be working on Windows and Apple. Linux still needs to be tested.

Diff Detail

Repository
rB Blender

Event Timeline

Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 9 2021, 12:56 PM

Successful build on Windows on my computer with the revision with the diff copy.

I don't think there is enough extension before considering multi-gpu queues.
A queue is basically something
used to upload data to the gpu,
What to use for compute shaders,
It can be divided into those used in the main graphics.
If it has overlays or XR, multiple graphic queues will be needed.

Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
  • Add compile option to include vulkan backend.
  • Add vulkan changes to cmake files.
  • Copied from tmp-vulkan branch.
  • Find MoltenVK (WIP).
  • Implemented createOffscreenContext.
  • Implemented newDrawingContext.
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Oct 21 2022, 1:04 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)

Successfully tested with gtest.
However, GHOST_SystemWin32.cpp, line2255 are commented out.

intern/ghost/intern/GHOST_SystemWin32.cpp
2287

This function depends on comctrl32.dll version 6 or higher. Otherwise segmentation fault with ordinal error.

  • Removed debug code.

I forked and built it here.
https://github.com/AgAmemnno/tmp-vulkan
I am testing with GTESTS of BF_GPU.
ContextVk's validation layer is mainly used for debugging, and vulkan has two extensions for that.
https://github.com/AgAmemnno/tmp-vulkan/blob/master/intern/ghost/intern/GHOST_ContextVK.cpp
So I tested the difference.
https://github.com/AgAmemnno/tmp-vulkan/blob/master/source/blender/gpu/tests/gpu_vulkan_basic_test.cc
Furthermore, if we add tags, Debug Report will not be supported.
Additionally, I ran Nvidia's python script at the cmake stage as it would load the function pointers.
line:81
https://github.com/AgAmemnno/tmp-vulkan/blob/master/intern/ghost/CMakeLists.txt

  • Add vulkan changes to cmake files.
  • Copied from tmp-vulkan branch.
  • Find MoltenVK (WIP).
  • Implemented createOffscreenContext.
  • Implemented newDrawingContext.
  • Removed obsolete comments.
  • Removed debug code.
  • Removed debug code.
  • Merge branch 'master' into temp-ghost-vulkan
  • Changes to cmake to select vulkan from libs.
  • Fix missing import in GHOST_SystemWin32.cpp
  • Finding MOLTENVK.
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 1 2022, 1:59 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
  • Create VKBackend when selecting vulkan from the command line.
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 1 2022, 2:54 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
Jeroen Bakker (jbakker) added inline comments.
intern/ghost/intern/GHOST_SystemWin32.cpp
2287

Thanks for the remark.

It isn't introduced by this patch so we should fix this upstream. Might want to discuss with developers what should happen with this code. on devtalk/blender chat.

Jeroen Bakker (jbakker) requested review of this revision.Nov 1 2022, 2:55 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 1 2022, 3:14 PM
intern/ghost/intern/GHOST_ContextVK.cpp
459

Geometry shader should not be required.

468
Ray Molenkamp (LazyDodo) added inline comments.
intern/ghost/intern/GHOST_SystemWin32.cpp
2287

We select the right version through blenders manifest, we haven't seen a comctrl32 related error since this code landed, was this ordinal error actually observed or is this comment just general warning of "watch out here!" ?

intern/ghost/intern/GHOST_SystemWin32.cpp
2287

I don't have version 6 of comctrl32.dll installed on my local computer, and
An ordinal error was observed by rewriting and building the ghost test cmake file.
It's not a canonical build, of course, so it's in the general sense.

Ray Molenkamp (LazyDodo) requested changes to this revision.Nov 1 2022, 5:22 PM

Every version of windows since XP has shipped with V6 or newer, but executables have to specifically request it (for backward compatibility reasons)

i'll take a closer look here, Requesting changes to keep this from landing until i can verify what is happening here.

This revision now requires changes to proceed.Nov 1 2022, 5:22 PM
intern/ghost/intern/GHOST_ContextVK.cpp
459

Yes, ideally. But we still don't have general purpose workaround for all of them. The metal backend reimplementation of the geometry shaders is using some tricks in the backend to bind VBOs as SSBOs, which I do not want in the Vulkan Backend.

468

This is only the dynamic state function. The baked pipeline option is here from vulkan 1.0 https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkPipelineColorBlendStateCreateInfo.html

And yes it is used, it is wrapped by GPU_logic_op_xor_set which could be changed to not require it. But finding another way to draw and keep the same clarity might be tricky. We did change the text edit cursors which lost a bit of clarity. While it might be nice to remove, implementing it is not that difficult.

CMakeLists.txt
1246–1249

Just by toggling WITH_VULKAN_BACKEND on/off in cmake, this will trigger a complete rebuild of blender due to the use of a global define, i think we can do better here.

intern/ghost/intern/GHOST_SystemWin32.cpp
2287

i don't quite understand what you mean, however with a stock build of this diff with vulkan enabled i cannot reproduce the issue you are seeing in either blender
or any of the test executables.

CMakeLists.txt
1246–1249

Will add them to the specific modules (ghost, window_manager and gpu) seems to be using it. When metal is a bit further we might want to do that also.

intern/ghost/intern/GHOST_SystemWin32.cpp
2287

What I mean is simple, it's just that ordinal errors are confusing. What is our policy when it comes to loading Vulkan extensions?

For example, I wrote a test for the difference between extensions in the validation layer for debugging.
Helpers for loading extensions had to be generated at the cmake stage.

line:81
https://github.com/AgAmemnno/tmp-vulkan/blob/master/intern/ghost/CMakeLists.txt

https://github.com/AgAmemnno/tmp-vulkan

intern/ghost/intern/GHOST_ContextVK.cpp
173

Remove void as argument in C++ files. Check other occurrences.

258
Ray Molenkamp (LazyDodo) resigned from this revision.Nov 3 2022, 6:36 PM

Given the ordinal thing appears to be a non issue, and i have nothing to add on the actual code, i'll resign from this one

This revision now requires review to proceed.Nov 3 2022, 6:36 PM

If you set the settings as follows
WITH_BUILDINFO=OFF,WITH_GTESTS=ON,WITH_VULKAN_BACKEND=ON,
An executable for bf_gpu_test is generated.
I wrote a simple flow of vulkan's render pipeline.
https://github.com/AgAmemnno/tmp-vulkan

intern/ghost/intern/GHOST_ContextVK.cpp
214

I wrote a pipeline test, but the submit loop causes a bug.
Shouldn't the fence work if this m_currentFrame is incremented?
Waiting for the fence immediately after submit was not a bug.
https://github.com/AgAmemnno/tmp-vulkan/blob/9afe5af6ed4fcb1882b80c23f76b8c15ca5bac90/intern/ghost/intern/GHOST_ContextVK.cpp#L907

intern/ghost/intern/GHOST_ContextVK.cpp
623

I think it's ambiguous whether to force reset the command buffer during the loop or conditionally reset it.
Not adding VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT to flags can cause bugs.

Validation Error: [ VUID-vkBeginCommandBuffer-commandBuffer-00050 ] Object 0: handle = 0x20e874b0d10,

type = VK_OBJECT_TYPE_COMMAND_BUFFER; 
Object 1: handle = 0x2e2cd000000002b, type = VK_OBJECT_TYPE_COMMAND_POOL; | MessageID = 0xb24f00f5 | 
Call to vkBeginCommandBuffer() on VkCommandBuffer 0x20e874b0d10[] attempts to implicitly reset cmdBuffer created from VkCommandPool 0x2e2cd000000002b[] that does NOT have the VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT bit set. 
The Vulkan spec states: 
If commandBuffer was allocated from a VkCommandPool which did not have the VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT flag set, 
commandBuffer must be in the initial state (https://vulkan.lunarg.com/doc/view/1.2.189.0/windows/1.2-extensions/vkspec.html#VUID-vkBeginCommandBuffer-commandBuffer-00050)
Jeroen Bakker (jbakker) marked 12 inline comments as done.
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
  • CMAKE: Move WITH_VULKAN_BACKEND from global to specific modules.
  • Remove (void) parameters from GHOST_ContextVK.
  • GHOST: Vulkan swapbuffer should wait for graphics queue idling.
  • Code style struct initialization.
  • GHOST: Command pool should be able to reset.
Jeroen Bakker (jbakker) added inline comments.
intern/ghost/intern/GHOST_ContextVK.cpp
214

This patch only contains some early development. Actual tests hasn't been performed yet. Would be awesome to have access to your pipeline test so we could already cover that part.
I applied the changes you mentioned.

intern/ghost/intern/GHOST_SystemWin32.cpp
2287

The policy for extension loading is still in discussion. The impact for this is also unclear
I haven't checked but doing this during preprocessing might lead to more licensing issues.
We already added a ticket for this T102055: Vulkan: Validation layers potential licensing risks. Any feedback from your experience is welcome.

This revision is now accepted and ready to land.Nov 22 2022, 10:44 AM
  • Merge branch 'master' into temp-ghost-vulkan
This revision was automatically updated to reflect the committed changes.
intern/ghost/intern/GHOST_ContextVK.cpp
214

When using Shaderc, I ported NVPRO's auxiliary class, but it is an early development for testing.
I used Spirv-cross when generating the pipeline layout. This has been integrated as an extern library.
Implemented Immediate draw test.
However, I only see shaders for doing icon_draw_rect.
Also, since the GPU_SHADER_3D_IMAGE_COLOR input is only 2D, I modified immDrawPixelsTexSetup and set "pos" to 3D.

No problem on my computer.

$make test
Start 40: bf_draw

40/248 Test #40: bf_draw ......................................... ................... Passed 1.19 sec

Please report any errors or crashes.
Source code: https://github.com/AgAmemnno/tmp-vulkan
I didn't follow up on your task enough, so there are many places where the writing doesn't match what you are committing right now.