Page MenuHome
Create Task
Maniphest T52021

Shadow Catcher bug with transparency and Branched Path Tracing
Closed, ResolvedPublic
Actions

Description

System Information

Ubuntu Linux 16.04, CPU rendering

Blender Version
Broken: (example: 2.78.5 ba256b32ee5)

Short description of error

When a transparent or semi-transparent material is used over a Shadow Catcher plane, the plane gets rendered in front of the transparent object when using Branched Path Tracing sampling.

Exact steps for others to reproduce the error

Just add a transparent object in front of a plane with Shadow Catcher enabled and use Branched Path Tracing sampling.

For example, this is the expected render obtained with Path Tracing sampling:

This is the broken one obtained with Branched Path Tracing:

This is the material and sampling settings used with the broken plane:

Revisions and Commits

rC Cycles
rCa3f6911e07c1 Fix T52021: Shadow catcher renders wrong when catcher object is behind…
rB Blender
rB5f35682f3abf Fix T52021: Shadow catcher renders wrong when catcher object is behind…

Related Objects

Event Timeline

Gustavo Niemeyer (niemeyer) created this task.Jul 11 2017, 4:44 AM
Gustavo Niemeyer (niemeyer) added a comment.Jul 11 2017, 4:46 AM

Setup for the example above:

shadow-catcher-bug.blend456 KBDownload

Sergey Sharybin (sergey) claimed this task.Jul 11 2017, 9:36 AM
Sergey Sharybin (sergey) lowered the priority of this task from 90 to 50.
Steffen Dünner (SteffenD) added a subscriber: Steffen Dünner (SteffenD).Jul 11 2017, 10:13 AM
Gustavo Niemeyer (niemeyer) added a comment.Jul 18 2017, 9:11 AM

Thanks for confirming the issue.

Is the fix for this likely to be in 2.79? The branched tracing greatly improves the rendering time in cases where a significant portion of the image is empty, which turns out to be a common case for the Shadow Catcher since it's integrating into existing footage.

Thanks again for looking into this.

Sergey Sharybin (sergey) added a comment.Jul 18 2017, 9:25 AM

The fix will be committed soon. Currently working on some unit tests to help testing shadow catcher in the future.

For the record, PT result was not correct either: if you render on a transparent film and alpha-over on an actual background you'll have artifacts. So both PT and BPT will be changed in render results.

Sergey Sharybin (sergey) changed the task status from Unknown Status to Resolved by committing rB5f35682f3abf: Fix T52021: Shadow catcher renders wrong when catcher object is behind….Jul 18 2017, 9:53 AM
Sergey Sharybin (sergey) added a commit: rB5f35682f3abf: Fix T52021: Shadow catcher renders wrong when catcher object is behind….
Gustavo Niemeyer (niemeyer) added a comment.Jul 18 2017, 4:18 PM

Thank you!

Sergey Sharybin (sergey) added a commit: rCa3f6911e07c1: Fix T52021: Shadow catcher renders wrong when catcher object is behind….Jul 25 2017, 12:08 PM
Gustavo Niemeyer (niemeyer) added a comment.Jul 28 2017, 1:46 AM

Is this supposed to be fixed in the latest builds? I'm still seeing artifacts in the nightly of 2.79 (more specifically, 2.78.5 1bd9531fda7).

I've reproduced by just putting a fully transparent object in front of a shadow catcher plane, and traces of the shadow remain on it.

I can try a new reproducer if that's useful.

Brecht Van Lommel (brecht) added a subscriber: Brecht Van Lommel (brecht).Jul 28 2017, 2:08 AM

The .blend file attached to this report seems to render correctly for me. If there's a different case that renders wrong, please do attach a .blend for that.

Gustavo Niemeyer (niemeyer) added a comment.Jul 28 2017, 2:35 AM

With that same example .blend file, just turn the factor all the way down to zero. The shadow (or AO, more likely) remain:

Gustavo Niemeyer (niemeyer) added a comment.Jul 30 2017, 6:44 PM

Here is a more clear example showing the issue and including the more noisy artifacts, in white background so it's easier to spot.

Rendered with 0.3 opacity it looks plausible (2.78.5 1bd9531fda7):

Going all the way down to zero shows the issue:

And this composites the phantom shadow and the extra noise on top of the prior scene:

Here is the blend file for the scene:

shadow-catcher-bug-2.blend497 KBDownload

Sergey Sharybin (sergey) added a comment.Jul 31 2017, 12:53 AM

@Gustavo Niemeyer (niemeyer), thanks for the detailed files. I am currently at siggraph, so really limited on time for bugfixing. But definitely looking into figuring this out.

What you might want to do is to copy the details to a new report dedicated to this particular problem. This way it'll make it easier to reference the issue from code and track changes back in the future when needed.

Sergey Sharybin (sergey) added a comment.Jul 31 2017, 2:02 AM

Here is a potential fix:

P518 Potential fix for transparency and shadow catcher
1diff --git a/intern/cycles/kernel/kernel_accumulate.h b/intern/cycles/kernel/kernel_accumulate.h
2index 9ed16aceb55..acfe1a5a63b 100644
3--- a/intern/cycles/kernel/kernel_accumulate.h
4+++ b/intern/cycles/kernel/kernel_accumulate.h
5@@ -223,6 +223,7 @@ ccl_device_inline void path_radiance_init(PathRadiance *L, int use_light_pass)
6 L->shadow_background_color = make_float3(0.0f, 0.0f, 0.0f);
7 L->shadow_radiance_sum = make_float3(0.0f, 0.0f, 0.0f);
8 L->shadow_throughput = 0.0f;
9+ L->shadow_transparency = 1.0f;
10 #endif
11
12 #ifdef __DENOISING_FEATURES__
13@@ -398,10 +399,11 @@ ccl_device_inline void path_radiance_accum_total_light(
14 #endif
15 }
16
17-ccl_device_inline void path_radiance_accum_background(PathRadiance *L,
18- ccl_addr_space PathState *state,
19- float3 throughput,
20- float3 value)
21+ccl_device_inline void path_radiance_accum_background(
22+ PathRadiance *L,
23+ ccl_addr_space PathState *state,
24+ float3 throughput,
25+ float3 value)
26 {
27 #ifdef __PASSES__
28 if(L->use_light_pass) {
29@@ -421,9 +423,7 @@ ccl_device_inline void path_radiance_accum_background(PathRadiance *L,
30 #ifdef __SHADOW_TRICKS__
31 if(state->flag & PATH_RAY_STORE_SHADOW_INFO) {
32 L->path_total += throughput * value;
33- if(state->flag & PATH_RAY_SHADOW_CATCHER_ONLY) {
34- L->path_total_shaded += throughput * value;
35- }
36+ L->path_total_shaded += throughput * value * L->shadow_transparency;
37 }
38 #endif
39
40@@ -671,7 +671,7 @@ ccl_device_inline float path_radiance_sum_shadow(const PathRadiance *L)
41 if(path_total != 0.0f) {
42 return path_total_shaded / path_total;
43 }
44- return 1.0f;
45+ return L->shadow_transparency;
46 }
47
48 /* Calculate final light sum and transparency for shadow catcher object. */
49diff --git a/intern/cycles/kernel/kernel_path.h b/intern/cycles/kernel/kernel_path.h
50index c340b3bc968..f2f59034bbb 100644
51--- a/intern/cycles/kernel/kernel_path.h
52+++ b/intern/cycles/kernel/kernel_path.h
53@@ -319,7 +319,8 @@ ccl_device void kernel_path_indirect(KernelGlobals *kg,
54
55 #ifdef __SHADOW_TRICKS__
56 if(!(sd->object_flag & SD_OBJECT_SHADOW_CATCHER)) {
57- state->flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY;
58+ L->shadow_transparency *=
59+ average(shader_bsdf_transparency(kg, sd));
60 }
61 #endif /* __SHADOW_TRICKS__ */
62
63@@ -644,7 +645,6 @@ ccl_device_inline float kernel_path_integrate(KernelGlobals *kg,
64 if((sd.object_flag & SD_OBJECT_SHADOW_CATCHER)) {
65 if(state.flag & PATH_RAY_CAMERA) {
66 state.flag |= (PATH_RAY_SHADOW_CATCHER |
67- PATH_RAY_SHADOW_CATCHER_ONLY |
68 PATH_RAY_STORE_SHADOW_INFO);
69 state.catcher_object = sd.object;
70 if(!kernel_data.background.transparent) {
71@@ -656,7 +656,8 @@ ccl_device_inline float kernel_path_integrate(KernelGlobals *kg,
72 }
73 }
74 else {
75- state.flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY;
76+ L->shadow_transparency *=
77+ average(shader_bsdf_transparency(kg, &sd));
78 }
79 #endif /* __SHADOW_TRICKS__ */
80
81diff --git a/intern/cycles/kernel/kernel_path_branched.h b/intern/cycles/kernel/kernel_path_branched.h
82index 77d4f1df447..48124b36095 100644
83--- a/intern/cycles/kernel/kernel_path_branched.h
84+++ b/intern/cycles/kernel/kernel_path_branched.h
85@@ -500,7 +500,6 @@ ccl_device float kernel_branched_path_integrate(KernelGlobals *kg,
86 #ifdef __SHADOW_TRICKS__
87 if((sd.object_flag & SD_OBJECT_SHADOW_CATCHER)) {
88 state.flag |= (PATH_RAY_SHADOW_CATCHER |
89- PATH_RAY_SHADOW_CATCHER_ONLY |
90 PATH_RAY_STORE_SHADOW_INFO);
91 state.catcher_object = sd.object;
92 if(!kernel_data.background.transparent) {
93@@ -511,7 +510,8 @@ ccl_device float kernel_branched_path_integrate(KernelGlobals *kg,
94 L->shadow_throughput = average(throughput);
95 }
96 else {
97- state.flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY;
98+ L->shadow_transparency *=
99+ average(shader_bsdf_transparency(kg, &sd));
100 }
101 #endif /* __SHADOW_TRICKS__ */
102
103diff --git a/intern/cycles/kernel/kernel_types.h b/intern/cycles/kernel/kernel_types.h
104index e6a62c42a38..e12557fc5b2 100644
105--- a/intern/cycles/kernel/kernel_types.h
106+++ b/intern/cycles/kernel/kernel_types.h
107@@ -346,8 +346,7 @@ enum PathRayFlag {
108 PATH_RAY_DIFFUSE_ANCESTOR = (1 << 13),
109 PATH_RAY_SINGLE_PASS_DONE = (1 << 14),
110 PATH_RAY_SHADOW_CATCHER = (1 << 15),
111- PATH_RAY_SHADOW_CATCHER_ONLY = (1 << 16),
112- PATH_RAY_STORE_SHADOW_INFO = (1 << 17),
113+ PATH_RAY_STORE_SHADOW_INFO = (1 << 16),
114 };
115
116 /* Closure Label */
117@@ -524,6 +523,7 @@ typedef ccl_addr_space struct PathRadiance {
118 */
119 float3 shadow_radiance_sum;
120 float shadow_throughput;
121+ float shadow_transparency;
122 #endif
123
124 #ifdef __DENOISING_FEATURES__
125diff --git a/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h b/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h
126index fec671be016..ad7e5d1cbee 100644
127--- a/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h
128+++ b/intern/cycles/kernel/split/kernel_holdout_emission_blurring_pathtermination_ao.h
129@@ -127,7 +127,6 @@ ccl_device void kernel_holdout_emission_blurring_pathtermination_ao(
130 if(state->flag & PATH_RAY_CAMERA) {
131 PathRadiance *L = &kernel_split_state.path_radiance[ray_index];
132 state->flag |= (PATH_RAY_SHADOW_CATCHER |
133- PATH_RAY_SHADOW_CATCHER_ONLY |
134 PATH_RAY_STORE_SHADOW_INFO);
135 state->catcher_object = sd->object;
136 if(!kernel_data.background.transparent) {
137@@ -143,7 +142,8 @@ ccl_device void kernel_holdout_emission_blurring_pathtermination_ao(
138 }
139 }
140 else {
141- state->flag &= ~PATH_RAY_SHADOW_CATCHER_ONLY;
142+ PathRadiance *L = &kernel_split_state.path_radiance[ray_index];
143+ L->shadow_transparency *= average(shader_bsdf_transparency(kg, sd));
144 }
145 #endif /* __SHADOW_TRICKS__ */
146

There are still some artifacts with less trivial topology than a box and AO-only lit scenes. Will need to have a closer look to see if that's something easily to solve.

P.S. If you don't mind, it's still handy to have separate report for this particular issue.

Gustavo Niemeyer (niemeyer) mentioned this in T52229: Shadow Catcher artifacts when under transparency.Jul 31 2017, 5:30 AM

@Sergey Sharybin (sergey) Thanks again for your attention on this.

I've create a more appropriate task as requested: T52229