Page MenuHome

Eevee Cryptomatte: Store hashes in render result meta data
ClosedPublic

Authored by Jeroen Bakker (jbakker) on Nov 13 2020, 4:26 PM.

Details

Summary

Stores cryptomatte hashes as meta data to the render result. Compositors could
use this for lookup on names in stead of hashes.

Cycles

Eevee

Diff Detail

Repository
rB Blender
Branch
arcpatch-D9553 (branched from master)
Build Status
Buildable 11584
Build 11584: arc lint + arc unit

Event Timeline

Jeroen Bakker (jbakker) requested review of this revision.Nov 13 2020, 4:26 PM
Jeroen Bakker (jbakker) created this revision.
Jeroen Bakker (jbakker) retitled this revision from EEVEE Cryptomatte: Store hashes in render result meta data to [WIP] EEVEE Cryptomatte: Store hashes in render result meta data.
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Nov 13 2020, 4:32 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)

Rebased with latest master.
Fixed name length issue (possible read out of bound)

Jeroen Bakker (jbakker) retitled this revision from [WIP] EEVEE Cryptomatte: Store hashes in render result meta data to EEVEE Cryptomatte: Store hashes in render result meta data.Dec 4 2020, 3:31 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
  • Refactored Cryptomatte Meta data
  • Added hash, conversion and manifest meta data
  • Replaced string manipulations to stringstream
  • Changed interface of add_hash method.
  • Removed unneeded include
  • Renames not empty to filled
  • Removed dynstr from cryptomatte module
  • Code style cleanup
Brecht Van Lommel (brecht) added inline comments.
source/blender/blenkernel/intern/cryptomatte.cc
69

The name string needs to be escaped, for when it contains e.g. \ or " characters (not sure about what else is needed, have not checked the JSON spec).

  • Escape (" or /) chars in manifest

@Jeroen Bakker (jbakker) I didn't see this until now. Apologies!

How can I help?

@Evan Wilson (EAW) no worries. The unknown is if the generated cryptomatte meta data can be used in external tools. I believe you have some experience (and access) to other packages. Just to have a check if a eevee rendered openexr file can be used in a similar way as cycles rendered openexr file.

source/blender/blenkernel/intern/cryptomatte.cc
69

remove double quotes

Jeroen Bakker (jbakker) marked an inline comment as not done.
  • Removed double quotes and whitespacing
Jeroen Bakker (jbakker) marked an inline comment as done.Dec 10 2020, 12:01 PM

Rebased with latest master

============================================================
Pipeline Summary - Linux
============================================================
Git Branch: arcpatch-D9553
Git Current Commit: 6d186c863c5117eb59509b24676f9f15831a6b5e
Git Parent Commit: 6e4fccd9fae37c20bc3b8ff52aeb5bf1328c56ad
Chassis: desktop
Operating System: Pop!_OS 20.04 LTS
Kernel: Linux 5.8.0-7630-generic
Architecture: x86-64
CPU Model: AMD Ryzen 9 3900X 12-Core Processor
Logical Cores: 24

step            elapsed          exitCode success
----            -------          -------- -------
clean-code      00:00:00.0482690        0    True
update-code     00:00:10.8024168        0    True
patch-code      00:00:09.3092154        0    True
compile-code    00:02:42.0794492        0    True
test-code-unit  00:02:46.7459723        0    True
test-code-smoke 00:00:00.0241617        0    True
============================================================
Pipeline Summary - Microsoft
============================================================
Git Branch: arcpatch-D9553
Git Current Commit: facca04d4ecaa99c71d8c31b59441b4f68821df6
Git Parent Commit: 6e4fccd9fae37c20bc3b8ff52aeb5bf1328c56ad
OS Detail: Microsoft Windows 10.0.18363
CPU Model: Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz
Logical Cores:  16

step         elapsed          exitCode success
----         -------          -------- -------
compile-code 00:00:45.2354866        1    False

============================================================
Error Summary - Microsoft
============================================================

step   : compile-code
errors : C:\Users\jmont\git\blender\blender.git\source\blender\blenkernel\intern\cryptomatte.cc(189,43): error C2065: 'NAME_MAX': undeclared identifier [C:\Users\jmont\git\blender\build\source\blender\blenkernel\bf_blenkernel.vcxproj]

Disregard crash on smoke test, running on VM

============================================================
Pipeline Summary - Darwin
============================================================
Git Branch: arcpatch-D9553_1
Git Current Commit: 0b1973d855d032ef6c73e8087a1e13ad1f757c87
Git Parent Commit: 6e4fccd9fae37c20bc3b8ff52aeb5bf1328c56ad
OS Detail: Darwin 19.6.0 Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64
CPU Model: Intel Core 2 Duo P9xxx (Penryn Class Core 2)
Logical Cores: 8
ProductName:    Mac OS X
ProductVersion: 10.15.7
BuildVersion:   19H15

step            elapsed          exitCode success
----            -------          -------- -------
clean-code      00:00:00.0008917        0    True
update-code     00:00:07.9476574        0    True
patch-code      00:00:11.0736968        0    True
compile-code    00:03:44.4115253        0    True
test-code-unit  00:05:22.7519513        0    True
test-code-smoke 00:00:00.2852579      139   False

============================================================
Error Summary - Darwin
============================================================

step   : compile-code
errors : 

step   : test-code-smoke
errors : Writing: /var/folders/kb/b1vjvq9s1zv2w4b142k6p0km0000gn/T/blender.crash.txt

Here is the call stack

# Blender 2.92.0, Commit date: 2020-12-11 14:12, Hash 0b1973d855d0

# backtrace
0   Blender                             0x0000000111b27097 BLI_system_backtrace + 55
1   Blender                             0x0000000109ddfba8 sig_handle_crash + 392
2   libsystem_platform.dylib            0x00007fff6eb745fd _sigtramp + 29
3   ???                                 0x0000000000000001 0x0 + 1
4   Blender                             0x00000001103994b4 _ZN7blender3gpu9GLBackend13platform_initEv + 84
5   Blender                             0x0000000110383a8c _ZN7blender3gpu9GLBackendC2Ev + 156
6   Blender                             0x000000011038381b GPU_context_create + 43
7   Blender                             0x000000010a22562d wm_window_ghostwindow_ensure + 381
8   Blender                             0x000000010a22549d wm_window_ghostwindows_ensure + 109
9   Blender                             0x000000010a1f4c23 WM_check + 179
10  Blender                             0x000000010a203802 wm_homefile_read + 1810
11  Blender                             0x000000010a20c5e9 WM_init + 329
12  Blender                             0x0000000109ddc3ca main + 778
13  libdyld.dylib                       0x00007fff6e977cc9 start + 1

# Python backtrace
============================================================
Pipeline Summary - Microsoft
============================================================
Git Branch: arcpatch-D9553
Git Current Commit: facca04d4ecaa99c71d8c31b59441b4f68821df6
Git Parent Commit: 6e4fccd9fae37c20bc3b8ff52aeb5bf1328c56ad
OS Detail: Microsoft Windows 10.0.18363
CPU Model: Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz
Logical Cores:  16

step         elapsed          exitCode success
----         -------          -------- -------
compile-code 00:00:45.2354866        1    False

============================================================
Error Summary - Microsoft
============================================================

step   : compile-code
errors : C:\Users\jmont\git\blender\blender.git\source\blender\blenkernel\intern\cryptomatte.cc(189,43): error C2065: 'NAME_MAX': undeclared identifier [C:\Users\jmont\git\blender\build\source\blender\blenkernel\bf_blenkernel.vcxproj]

^@Jeroen Bakker (jbakker) This is what is currently preventing me from testing this patch.

source/blender/blenkernel/intern/cryptomatte.cc
190

NAME_MAX to MAX_NAME

  • Fixed compile error on windows
============================================================
Pipeline Summary - Microsoft
============================================================
Git Branch: arcpatch-D9553_2
Git Current Commit: 2bda6db7fc91569ec7c8646761894b686b38368d
Git Parent Commit: 6e4fccd9fae37c20bc3b8ff52aeb5bf1328c56ad
CPU Model: Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz
Logical Cores:  16
GPU Model: Intel(R) UHD Graphics
GPU Model: NVIDIA GeForce RTX 2070 Super with Max-Q Design

step            elapsed          exitCode success
----            -------          -------- -------
clean-code      00:00:00.1992820        0    True
update-code     00:00:05.9956911        0    True
patch-code      00:00:15.1822494        0    True
compile-code    00:33:41.6806633        0    True
test-code-unit  00:06:06.3511395        0    True
test-code-smoke 00:00:07.7692346        0    True

============================================================
Error Summary - Microsoft
============================================================

step   : compile-code
errors :

step   : test-code-smoke
errors :



============================================================
Warning Summary - Microsoft
============================================================

step     : compile-code
warnings : {C:\Users\jmont\git\blender\blender.git\source\blender\blenkernel\intern\layer.c(2088,62): warning C4028: formal parameter 7 different    
           from declaration [C:\Users\jmont\git\blender\build\source\blender\blenkernel\bf_blenkernel.vcxproj],
           C:\Users\jmont\git\blender\blender.git\intern\cycles\render\scene.cpp(449,74): warning C4805: '|': unsafe mix of type 'bool' and type 
           'int' in operation [C:\Users\jmont\git\blender\build\intern\cycles\render\cycles_render.vcxproj]}

step     : test-code-smoke
warnings :
Evan Wilson (EAW) added a comment.EditedDec 13 2020, 4:18 AM

The manifest metadata isn't getting written to the file, at least not on my machine.
Setting a breakpoint at the end of cryptomatte.cc shows that the name, hash, conversion, and manifest are all there as expected in BKE_render_result_stamp_data, but when looking at the header metadata in the saved EXR file, the manifest isn't there.
The same build correctly saves the manifest metadata from Cycles.

I will have a look. Could be that it is stored on the copy and not synced back to the final render result.

Rebased with latest master

Jeroen Bakker (jbakker) planned changes to this revision.Dec 15 2020, 5:43 PM

Test if Meta data are stored in the Multilayer OpenEXR file

Jeroen Bakker (jbakker) retitled this revision from EEVEE Cryptomatte: Store hashes in render result meta data to Eevee Cryptomatte: Store hashes in render result meta data.Dec 16 2020, 10:56 AM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
  • Use the correct cryptomatte name
  • Fixed intermediate '\0' in cryptomatte name

Added store_metadata callback to draw engine type.

This revision was not accepted when it landed; it landed in state Needs Review.Jan 5 2021, 3:04 PM
This revision was automatically updated to reflect the committed changes.