Page MenuHome

Fix T64953: Add cryptomatte meta data to file output node.
ClosedPublic

Authored by Jeroen Bakker (jbakker) on Jan 6 2021, 1:48 PM.

Details

Summary

This change will try to add meta data when using a multilayered open
exr file output node in the compositor. It adds the current scene meta
data and converts existing cryptomatte keys so it follows the
naming that is configured in the file output node.

This change supports the basic use-case where the compositor is
used to output cryptomatte layers with a different naming scheme to
support external compositors. In this case the Multilayered OpenEXR
files are used and the meta data is read from the render result.

Meta data is found when render layer node is connected with the
file output node without any other nodes in between. Redirects and empty
node groups are allowed.

The patch has been verified to work with external compositors.
See https://devtalk.blender.org/t/making-sense-of-cryptomatte-usage-in-third-party-programs/16576/17

Example files

Example blend + openexr file with multiple view layers.

As an additional bonus the stamp data of the active scene is also
outputted in the file.

Diff Detail

Repository
rB Blender
Branch
T64953-file-output-node-meta-data (branched from master)
Build Status
Buildable 12051
Build 12051: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Jan 8 2021, 12:01 PM
Jeroen Bakker (jbakker) marked 10 inline comments as done.Jan 8 2021, 1:59 PM

Updated patch with comments from code review:

  • Use blender::StringRef.
  • Use constexpr.
  • Added description in header file.
  • Added lines breaks in various locations.
  • Removed using namespace blender;

Patch has also been verified to work with external compositors.

Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Jan 8 2021, 2:05 PM
  • Remove extra references of StringRef parameters
  • Use StringRefs in COM_MetaData
  • Reduce string copies in COM_RenderLayersProg
source/blender/blenkernel/BKE_cryptomatte.hh
34

Strange enough when I commit it has an end-of-line.

source/blender/compositor/operations/COM_OutputFileOperation.cpp
337–338

If this is just a useless temporary, creating it on stack will not need a free later on. Though I may be misunderstanding its use. I saw this because of construction using MEM_callocN instead of new.

345–346

This passing a temporary std::string to StringRef -> returning a StringRef -> creating a new std::string seems suspicious. Would the temporary outlive the function call, or would there be a dangling pointer ?

Also, std::string will count until \0 anyway, so using BLI_strnlen is duplicating the work..
https://godbolt.org/z/o3fW87

I'd say pass the layer->name directly to BKE_cryptomatte_extract_layer_name, get back a StringRef and pass it to other functions later on.

source/blender/compositor/operations/COM_RenderLayersProg.cpp
228–230

Just checking if a delete corresponding to this new is needed or if it's intentional to omit the destructor.

241–243

std::string has a constructor with const char * etc. A copy construction from another std::string is very likely not needed.
Same below. https://en.cppreference.com/w/cpp/string/basic_string/basic_string

Oops I hadn't refreshed

source/blender/blenkernel/BKE_cryptomatte.hh
58

Why mark return value const ? Anyone accessing the string through StringRef will not be able to modify the original string anyway.

source/blender/blenkernel/intern/cryptomatte.cc
304–308

Reference is not needed.. it's cheap to pass these around.

source/blender/compositor/operations/COM_RenderLayersProg.cpp
247

Same probably now applies to blender::StringRefNull key = blender::StringRefNull(propname);
Consider blender::StringRefNull key(propname);

  • Fixed compile error on Windows VC16
  • Rebased with master
  • Documented the 7 char hash limitation
Jeroen Bakker (jbakker) marked 7 inline comments as done.
  • Removed const from BKE_cryptomatte_extract_layer_name return value.
  • Changed StringRef& => StringRef
  • Allocate temp RenderResult on stack.
  • Removed unneeded StringRef->string->StringRef conversion.
  • Added comments explaining the life time of the return parameter of getMetaData.
  • Less is more. Use short initialization when constructing StringRef field on stack.
source/blender/blenkernel/intern/cryptomatte.cc
304–308

Yes I missed these during cleanup.

source/blender/compositor/intern/COM_MetaData.h
41

Change to class, it does not compile using vc16.

source/blender/compositor/operations/COM_OutputFileOperation.cpp
345–346

To my understanding DNA char-arrays containing names are null terminated, except when they use the whole space. Therefore you always need to use BLI_strnlen otherwise it will construct an incorrect string.

I cleaned up the rendundant transformations.

source/blender/compositor/operations/COM_RenderLayersProg.cpp
228–230

I added some comments about the lifetime of the MetaData instance.

241–243

I kept to the C-function definition as it is a C callback. The constructor would not be called from C -> CPP as the caller is responsible for calling the constructor.

👍

source/blender/compositor/operations/COM_OutputFileOperation.cpp
345–346

Oh, didn't know that.

source/blender/compositor/operations/COM_RenderLayersProg.cpp
241–243

Scratch this comment. It moved up because I had not refreshed while you changed the patch.
It referred to std::string key = std::string(propname); (which is now gone).

Nice to see demo file! Question though, is how do you verify that the result file is all fine?

You ask users to test of course https://devtalk.blender.org/t/making-sense-of-cryptomatte-usage-in-third-party-programs/16576/12 .

I can confirm that the example .exr file works fine in Nuke v14 (asset metadata was not included, but both Object and Material layer were correct), and so far other people in the thread have tried the same file also in fusion16 and photoshop, and it seems it works fine as well.

Sergey Sharybin (sergey) requested changes to this revision.Jan 11 2021, 9:54 AM

You ask users to test of course https://devtalk.blender.org/t/making-sense-of-cryptomatte-usage-in-third-party-programs/16576/12 .

As a first step, sure. After that you add regression test to the compositor module ;)

source/blender/blenkernel/intern/cryptomatte.cc
365

Cover the new functions with tests.

You can use rB401612b8e14 to see how easy it is to add new tests.

source/blender/compositor/intern/COM_MetaData.cpp
69

@Jacques Lucke (JacquesLucke) Is it something what was decided to do in Blender now? Can it be more explicitly typed?

source/blender/compositor/intern/COM_MetaData.h
32–35

Does {hash}in the string have some special meaning?

43

Trailing underscore for private members (see the `Class data member names
` section of Coding Style (C/C++))

source/blender/compositor/operations/COM_RenderLayersProg.cpp
221

My guess is that for proper lazy initialization you need to have MetaData *meta_data = nullptr;

228–230

I can not really understand the comment.

Why it needs to be lazy-initialized? Are there a lot of cases when callback object is created but addMetaData is never called?

Compositor operates in a multi-threaded environment, and this class is not thread-safe. What governs thread-safety? Add a note in the comment to this class.

Callers to this method are responsible to remove the instance.

This method == addMetaData ? Which instance?

The part of the comment about getMetaData and addMetaData I don't really understand, so can not give suggestions about how to make it more clear at this time.

source/blender/compositor/operations/COM_RenderLayersProg.h
97

Commit such cleanup separately. Can go straight into master.

125

Implicit (and suggested by Clang-Tidy) is: MetaData *getMetaData() const override; Makes it clear that this is a virtual call.

Some folks prefers more explicit way of virtual MetaData *getMetaData() const override;

source/blender/compositor/operations/COM_SocketProxyOperation.h
44

So now I guess comment in the CallbackData from above makes more sense from information point of view, but it is done in the wrong place.

If the caller of SocketProxyOperation::getMetaData is to destroy the the result of getMetaData() it must be written down as a comment to this method.

It also seems it will be more clear and correct to do it as unique_ptr<MetaData> getMetaData() const; (and replace allocation with make_unique<> to fully switch to the more modern way of dealing with smart pointers). In this case you don't even need to trouble of:

  • Writing up that the result is to be destroyed
  • Hugely reduces risk of memory leak (leak is still possible if you do something "illegal", but that should be quite easy to catch on review).
This revision now requires changes to proceed.Jan 11 2021, 9:54 AM
Jeroen Bakker (jbakker) marked 6 inline comments as done.
  • Removed auto with actual type.
  • Added override to overriden virtual methods.
  • Use std::unique_ptr<MetaData> as return type for getMetaData.
Jeroen Bakker (jbakker) planned changes to this revision.Jan 11 2021, 12:41 PM

Add test cases. Not 100% sure how much work the end to end test as it needs to create a temp file and load it back in and test the meta data of the loaded file.

source/blender/compositor/intern/COM_MetaData.cpp
69

Not sure if it is decided. Just copied it from the test case that explained how to use the map. Can use the correct data type here.

source/blender/compositor/operations/COM_OutputFileOperation.cpp
347

This .get() is not necessary. You can use the -> operator directly on the std::unique_ptr.

  • Replaced .get()-> with -> for unique_ptr
  • Added BKE Cryptomatte TestCases
  • Added BKE Cryptomatte TestCases

After that you add regression test to the compositor module ;)

Regression test for this topic is complex to do:

  1. Render tests do not test meta data.
  2. bpy.types.Image does not support metadata.
  3. bpy.types.Movieclip supports metadata, but only for ffmpeg movie files, not for image sequences.
  4. Implementing the test case in CPP would not be the right direction.

I would prefer to hold off the overall test and implement a metadata retrieval for image files T84598: Add Python API for Image meta data.

1diff --git a/tests/python/CMakeLists.txt b/tests/python/CMakeLists.txt
2index f33bb81d6b0..232c9b36564 100644
3--- a/tests/python/CMakeLists.txt
4+++ b/tests/python/CMakeLists.txt
5@@ -785,3 +785,12 @@ add_subdirectory(collada)
6
7 # TODO: disabled for now after collection unification
8 # add_subdirectory(view_layer)
9+
10+
11+# COMPOSITOR TESTS
12+if(WITH_COMPOSITOR AND WITH_CYCLES)
13+ add_blender_test(
14+ script_run_operators
15+ --python ${CMAKE_CURRENT_LIST_DIR}/bl_compositor.py
16+ )
17+endif()
18\ No newline at end of file
19diff --git a/tests/python/bl_compositor.py b/tests/python/bl_compositor.py
20new file mode 100644
21index 00000000000..303b5b95c5b
22--- /dev/null
23+++ b/tests/python/bl_compositor.py
24@@ -0,0 +1,51 @@
25+# Apache License, Version 2.0
26+
27+# ./blender.bin --background -noaudio --python tests/python/bl_compositor.py -- --verbose
28+import bpy
29+import unittest
30+import random
31+
32+
33+class TestCompositor(unittest.TestCase):
34+ def test_file_output_node_meta_data(self):
35+ # Setup render engine
36+ scene = bpy.context.scene
37+ scene.render.engine = 'CYCLES'
38+ scene.render.resolution_x = 256
39+ scene.render.resolution_y = 256
40+ scene.cycles.samples = 8
41+ scene.frame_current = 1
42+ # Setup view layer
43+ view_layer = bpy.context.view_layer
44+ view_layer.use_pass_cryptomatte_object = True
45+ # Setup compositor
46+ scene.use_nodes = True
47+ node_tree = scene.node_tree
48+ render_layers_node = node_tree.nodes['Render Layers']
49+ file_output_node = node_tree.nodes.new("CompositorNodeOutputFile")
50+ file_output_node.format.file_format = 'OPEN_EXR_MULTILAYER'
51+ file_output_node.base_path = bpy.app.tempdir+"cryptomatte_"
52+ file_output_node.layer_slots[0].name = "CryptomatteLayer00"
53+ file_output_node.layer_slots.new("CryptomatteLayer01")
54+ file_output_node.layer_slots.new("CryptomatteLayer02")
55+
56+ node_tree.links.new(render_layers_node.outputs["CryptoObject00"],
57+ file_output_node.inputs[0])
58+ node_tree.links.new(render_layers_node.outputs["CryptoObject01"],
59+ file_output_node.inputs[1])
60+ node_tree.links.new(render_layers_node.outputs["CryptoObject02"],
61+ file_output_node.inputs[2])
62+ # Render + Composite + Write File
63+ bpy.ops.render.render()
64+
65+ openexr_file_path = bpy.app.tempdir+"cryptomatte_0001.exr"
66+ image = bpy.data.images.load(filepath=openexr_file_path, check_existing=True)
67+ # TODO: how to check the meta data of the generated file.
68+ # Movieclip has a `metadata` function but only supports ffmpeg meta data.
69+ # Image has no access to the `metadata` from Python API.
70+
71+
72+if __name__ == '__main__':
73+ import sys
74+ sys.argv = [__file__] + (sys.argv[sys.argv.index("--") + 1:] if "--" in sys.argv else [])
75+ unittest.main()
contains an attempt to implement it in Python.

Jeroen Bakker (jbakker) marked 4 inline comments as done.
  • Added comment about the {hash} substrings used in the meta data keys
  • Added trailing underscore to private class data members

@Jeroen Bakker (jbakker), so it is not enough to compare image output?

@Jeroen Bakker (jbakker), so it is not enough to compare image output?

idiff only compares pixels, not meta data.

Ok then, lets keep that for later then since we can't do such test easily yet.

Just remove the move() from return statement and should be fine. You can read up about it in the internet, or trust strict compiler flags to suggest to remove move from return :)

source/blender/compositor/intern/COM_SocketReader.h
142

A bit redundant, but probably fine.

source/blender/compositor/operations/COM_RenderLayersProg.cpp
293

Don't use move in return, as it breaks return value optimization.

Jeroen Bakker (jbakker) marked an inline comment as done.
  • Remove redundant comment
source/blender/compositor/operations/COM_RenderLayersProg.cpp
293

I think that this isn't the case here as the unique_ptr is owned by callback_data. When removing std::move here I get compiler errors. Perhaps you had a different approach in mind?

/home/jeroen/blender-git/blender/source/blender/compositor/operations/COM_RenderLayersProg.cpp: In member function ‘virtual std::unique_ptr<MetaData> RenderLayersProg::getMetaData() const’:
/home/jeroen/blender-git/blender/source/blender/compositor/operations/COM_RenderLayersProg.cpp:293:24: error: use of deleted function ‘std::unique_ptr<_Tp, _Dp>::unique_ptr(const std::unique_ptr<_Tp, _Dp>&) [with _Tp = MetaData; _Dp = std::default_delete<MetaData>]’
  293 |   return callback_data.meta_data;
      |                        ^~~~~~~~~
In file included from /usr/include/c++/9/memory:80,
                 from /home/jeroen/blender-git/blender/source/blender/blenlib/BLI_memory_utils.hh:26,
                 from /home/jeroen/blender-git/blender/source/blender/blenlib/BLI_array.hh:42,
                 from /home/jeroen/blender-git/blender/source/blender/blenlib/BLI_map.hh:72,
                 from /home/jeroen/blender-git/blender/source/blender/compositor/intern/COM_MetaData.h:23,
                 from /home/jeroen/blender-git/blender/source/blender/compositor/intern/COM_SocketReader.h:22,
                 from /home/jeroen/blender-git/blender/source/blender/compositor/intern/COM_MemoryBuffer.h:25,
                 from /home/jeroen/blender-git/blender/source/blender/compositor/intern/COM_NodeOperation.h:29,
                 from /home/jeroen/blender-git/blender/source/blender/compositor/operations/COM_RenderLayersProg.h:23,
                 from /home/jeroen/blender-git/blender/source/blender/compositor/operations/COM_RenderLayersProg.cpp:19:
/usr/include/c++/9/bits/unique_ptr.h:414:7: note: declared here
  414 |       unique_ptr(const unique_ptr&) = delete;
      |       ^~~~~~~~~~
Sergey Sharybin (sergey) added inline comments.
source/blender/compositor/operations/COM_RenderLayersProg.cpp
293

It might be, actually.

Leave it in then :)

This revision is now accepted and ready to land.Jan 12 2021, 4:23 PM