Paths

Table of Contentst

Cycles X: Initial implementation of shadow catcher
ClosedPublic
Actions

Authored by Sergey Sharybin (sergey) on May 5 2021, 7:06 PM.

Details

Reviewers

Summary

It is re-implemented in a way which does differential rendering: the
path is split at a first non-transparent bounce when shadow catcher
object is hit. One path will track state of the shadow catcher object
without other objects affecting it. The other path is rendering scene
as usual.

Upon user access of the shadow catcher pass the result is calculated
as combined pass divided by the pass with shadow catcher object. This
gives a pass which is to be multiplied with a backdrop to transport
light and shadows caught.

For the artificial objects which are to be added to the backdrop
there is an internal matte pass. This pass is presented as combined
to artists, so they alpha-over it onto the footage as usual. In the
future we might implement shadow approximation to gain old-style
shadow catcher behavior (maybe for compatibility reasons, maybe for
some artistic reasons).

This process is a bit more involved that the old single pass approach,
but it allows to deal with indirect light, caustics.

Example file:

fabrik_catcher.blend1 MBDownload

Example result:

The footage is the Old_Factory MVI_4005.mov from

https://cloud.blender.org/training/track-match-blend/56040f9b044a2a00ad6c660d

Diff Detail

Repository

rB Blender

Branch

cycles-x-shadow_catcher_v3 (branched from master)

Build Status

Buildable 14436
Build 14436: arc lint + arc unit

Event Timeline

Sergey Sharybin (sergey) requested review of this revision.May 5 2021, 7:06 PM

Sergey Sharybin (sergey) created this revision.

Harbormaster completed remote builds in B14393: Diff 36803.May 5 2021, 7:06 PM

Sergey Sharybin (sergey) planned changes to this revision.May 5 2021, 7:07 PM

Sergey Sharybin (sergey) edited the summary of this revision. (Show Details)May 5 2021, 7:15 PM

Brecht Van Lommel (brecht) awarded a token.May 5 2021, 8:31 PM

Fran Marz (franMarz) awarded a token.May 5 2021, 11:50 PM

Alaska (Alaska) awarded a token.May 6 2021, 10:14 AM

For the passes, I think what is currently the "Shadow Catcher Matte" can replace the Combined pass? I'm not sure if there is a need to have that as its own pass, from the point of view of users I'd expect that object to immediately turn transparent when enabling Shadow Catcher.

For naming, I'm not sure what the best direction is. First question is if this feature should still be called "Shadow Catcher" for users, since it does more than shadows now. I see other renderers using terms "Shadow Catcher", "Shadow/Reflection Collector", "Shadow/Reflection Matte", "Shadow Matte", "Matte". None of them are much better in my opinion, so we might stick with what we have unless someone thinks of a better term.

Internally I think "split path" can be "catcher path". If it becomes a generic mechanism we can rename it again.

intern/cycles/kernel/device/cuda/kernel.cu
265–270	I guess it's possible to halve `num_states` and the work size on the host side, and not do the test here.
271–275	The same is needed in `kernel_cuda_integrator_active_paths_array`, for the megakernel.
intern/cycles/kernel/integrator/integrator_path_state.h
110–112 ↗	(On Diff #36803)	This may be needed for the shadow path as well?
154–156 ↗	(On Diff #36803)	Same comment.

For the passes, I think what is currently the "Shadow Catcher Matte" can replace the Combined pass?

Think so too. We only need combined pass for the division. Don't think it will be of any use for users.
Was thinking of doing it in the pass_accessor.cpp, so that the kernel logic stays the same.

The name I share your opinion as well. So lets hold off with changes in them :)

Internally I think "split path" can be "catcher path". If it becomes a generic mechanism we can rename it again.

Ok, will do in the next iteration of the patch.

intern/cycles/kernel/device/cuda/kernel.cu
265–270	Yeah, we can limit `num_states` on the host. Although, is not really half since it could be 10 "regular" states and 5 shadow catcher states. But that's details.
271–275	Not entirely sure. The active paths gives indices of states which are still alive, so both "regular" and shadow catcher states are to be counted there (so that they all will finish). The terminated paths array is what seems to be used for re-scheduling. ad that's where (at least for now) it seemed important to (a) never schedule "regular" path into the high half of the states array and (b) don't reschedule "regular" path if its complementary shadow catcher state is still active. Although, I might be missing something.
intern/cycles/kernel/integrator/integrator_path_state.h
110–112 ↗	(On Diff #36803)	Good point!

Brecht Van Lommel (brecht) added inline comments.May 6 2021, 6:05 PM

intern/cycles/kernel/device/cuda/kernel.cu
271–275	Ah, I think you're right.

stefano severi (Tetone) awarded a token.May 6 2021, 6:32 PM

Brecht Van Lommel (brecht) added inline comments.May 6 2021, 7:30 PM

intern/cycles/kernel/integrator/integrator_intersect_closest.h
137–138	This is missing `INTEGRATOR_PATH_SET_SORT_KEY` for the split path. It's used for sorting by shader before shade_surface. I'm not sure if that will explain a negative counter. But it's possible if there is memory that happens to have value `CUDA_PARALLEL_SORTED_INDEX_INACTIVE_KEY`.

Brecht Van Lommel (brecht) added inline comments.May 6 2021, 7:49 PM

intern/cycles/kernel/integrator/integrator_intersect_closest.h
137–138	Actually the value in `__integrator_sort_key_counter` will be off too, that's a more likely reason this can go wrong.

Brecht Van Lommel (brecht) mentioned this in D11185: Cycles X: refactoring of kernel globals.May 6 2021, 9:22 PM

Paul Larson (GeorgiaPacific) added a subscriber: Paul Larson (GeorgiaPacific).May 7 2021, 2:38 AM

Taras (tako) awarded a token.May 7 2021, 8:57 AM

Thanks Brecht! I'll give it a whirl ASAP and update the patch with as many changes/fixes mentioned here as possible for today :)

Alistair Sealy (alisealy) added a subscriber: Alistair Sealy (alisealy).May 7 2021, 2:03 PM

Update against latest cycles-x branch
Combined pass is now the one which is to be alpha-overed
Support for CUDA and OptiX
I think inlined comments are addressed, but please double-check!

Overall I think we are the home stretch of getting it committed! :)

Harbormaster completed remote builds in B14436: Diff 36867.May 7 2021, 4:13 PM

Sergey Sharybin (sergey) edited the summary of this revision. (Show Details)May 7 2021, 4:14 PM

Sergey Sharybin (sergey) retitled this revision from Cycles X: WIP, Initial implementation of shadow catcher to Cycles X: Initial implementation of shadow catcher.

Brecht Van Lommel (brecht) added inline comments.May 7 2021, 4:31 PM

intern/cycles/kernel/integrator/integrator_state_flow.h
114–116	Only do this when `(path, queued_kernel) == SHADE_SURFACE`.
115	This is no longer needed now that key is part of `integrator_state_template.h`.

Addressed new inlined comments from Brecht.

Also, run our benchmark to verify there is no performance loss:

                              shadow_catcher                cycles-x
bmw27.blend                   10.3184                       10.3558
classroom.blend               16.5258                       16.57
pabellon.blend                9.08205                       9.077
monster.blend                 11.991                        11.9629
barbershop_interior.blend     12.4519                       12.5076
junkshop.blend                16.4619                       16.5284
pvt_flat.blend                17.2813                       17.2212

Harbormaster completed remote builds in B14440: Diff 36873.May 7 2021, 5:01 PM

Looks good for the first version.

This revision is now accepted and ready to land.May 7 2021, 5:10 PM

Thanks for the review!
Committed at rBc77529091f8.

HEYPictures (HEYPictures) awarded a token.May 8 2021, 11:51 AM

filip mond (vklidu) awarded a token.May 10 2021, 12:17 PM

Steffen Dünner (SteffenD) added a subscriber: Steffen Dünner (SteffenD).May 10 2021, 8:14 PM

Stas (girafic) awarded a token.May 10 2021, 10:46 PM

Konstantins Visnevskis (kivig) awarded a token.May 12 2021, 12:55 AM

Martin Malmqvist (Malmqvist) awarded a token.May 12 2021, 11:17 AM

Christian Brinkmann (poor) awarded a token.Jun 21 2021, 4:09 PM

Christian Brinkmann (poor) added a subscriber: Christian Brinkmann (poor).