Details

Reviewers

Clément Foucault (fclem)
Brecht Van Lommel (brecht)

Maniphest Tasks

T62876: Camera's background image invisible in wireframe mode

Commits

rBfed6c1a970f1: Fix T62876: Camera Background Images

Summary

Migrate old legacy code to the draw mamager/object mode. The old legacy
version did not work with wireframe. By migrating the code
to modern draw manager code we have mode control on the drawing process.

Still background images do not work with OIT, the cause seems to be that the transparent pixels are treated as background pixels. The next picture makes this clear.

Fix: T62876, T63237

Diff Detail

Repository: rB Blender

Event Timeline

Jeroen Bakker (jbakker) created this revision.Apr 3 2019, 3:08 PM

Harbormaster completed remote builds in B3268: Diff 14548.Apr 3 2019, 3:08 PM

WIP: Refactor Camera Background Images

Migrating legacy drawing code for camera background images
to be compatible with draw manager.

two questions still needs to be validated:

Is there a better way to calculate the matrices. I simulate a screen/viewrect space transformation.
GPU textures of the MovieClip are not freed when scrubbing this leads to memory issues. Need a clever way for freeing unused images.

Harbormaster completed remote builds in B3269: Diff 14550.Apr 3 2019, 3:11 PM

Jeroen Bakker (jbakker) retitled this revision from Camera Background Images to WIP: Camera Background Images.Apr 3 2019, 3:17 PM

Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)

Jeroen Bakker (jbakker) added inline comments.Apr 3 2019, 3:26 PM

source/blender/draw/modes/object_mode.c
1182	update comment
1354	remove line
source/blender/draw/modes/shaders/object_camera_image_frag.glsl
7	Add depth to the gl_Position?
source/blender/draw/modes/shaders/object_camera_image_vert.glsl
14	Better names for pos_s, pos2d
19	in stead of booleans we can use a vec2.

Jeroen Bakker (jbakker) added inline comments.Apr 3 2019, 3:51 PM

source/blender/draw/modes/object_mode.c
1289	Change to vec2
source/blender/gpu/intern/gpu_draw.c
247	Merge with `gpu_get_image_gputexture` code

Use premul for in front rendering and use OIT blending for rendering in back.
As rendering in back is additive the visual results differ from what users
might expect when changing the alpha value.

Issue is related that IOT transparent objects have an alpha value of 0.0 in the DST buffer.
This will give other artifacts (transparent pixels are seens as background and overwritten at all.

This is a shortcoming of having all these options to help the user and how we render them in the pipeline.

Harbormaster completed remote builds in B3274: Diff 14573.Apr 4 2019, 12:11 PM

Set background images to use premul blending.

Harbormaster completed remote builds in B3275: Diff 14576.Apr 4 2019, 12:38 PM

Removed premul

Harbormaster completed remote builds in B3276: Diff 14577.Apr 4 2019, 12:40 PM

Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Apr 4 2019, 12:46 PM

Jeroen Bakker (jbakker) added reviewers: Clément Foucault (fclem), Brecht Van Lommel (brecht).

Jeroen Bakker (jbakker) retitled this revision from WIP: Camera Background Images to Camera Background Images.Apr 10 2019, 12:56 PM

Rebased with master (clang-format)

Harbormaster completed remote builds in B3347: Diff 14773.Apr 17 2019, 10:35 AM

Rebased with latest master. Due to recent changes the color managament of the bg pics needs to be reviewed.
The GPU_textures are scene_linear. So we need to perform the view transform, before performing the blending
to get similar results as before.

This might get us one step back.

Harbormaster completed remote builds in B3889: Diff 15950.Jun 18 2019, 1:56 PM

I think this would just copy the code from object_empty_image_frag.glsl for color management?

Added color management

Harbormaster completed remote builds in B3893: Diff 15954.Jun 18 2019, 3:37 PM

Jeroen Bakker (jbakker) added inline comments.Jun 18 2019, 3:39 PM

source/blender/draw/modes/object_mode.c
1282	Revisit this order, rotation and translation do not work together.

Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Jun 18 2019, 3:45 PM

Fixed matices

source/blender/draw/modes/object_mode.c
1282	Also scaling does not work as expected.

Harbormaster completed remote builds in B3895: Diff 15958.Jun 19 2019, 10:35 AM

@Jeroen Bakker (jbakker) Holdout in combination with transparent film doesn't seem to work. When scene as Film Transparent enabled and I create a viewlayer that is set to holdout, the Cycles viewport rendering shows the transparent area created by the holdout as grey world background, but not the Background images. You can try this file:

camera_background_images.blend140 KBDownload

Just hit shift+z to go to rendered viewport to see the issue.
It does work fine in 2.79 (using mask layer in renderlayers)

@Jeroen Bakker (jbakker) DOF in Cycles over background images doesn't work at all:

Sean Kennedy (hype) added a subscriber: Sean Kennedy (hype).Jun 20 2019, 7:44 AM

I see that holdout is not working as expected. You can say it was not working before as holdout influences the Z-buffer and therefore also influences the grid rendering and viewport rendering on multiple places.

This is current master, showing that the grid lines is hidden by the holdout.

When film is not transparent background images are also not drawn

The new way of camera background rendering uses Z-buffer to test how to integrate the camera background images, hence the holdout effects the image rendering.
The depth buffer needs to be correct, otherwise the viewport compositing will not work.

Hah, this patch supports camera background image rendering in cycles without film transparency enabled. You win some, you loose some.

A possible solution might be to use an Alpha test before depth test, will experiment with this.

Hmmm seems that we don't have actual alpha values to play with so there will always be artifacts....

I think we can go ahead and commit this change, assuming @Clément Foucault (fclem) approves. It's better than not showing background images at all.

Ideally it would probably first draw with the render engine with proper alpha channel, then alpha under the background images, and then alpha under the checkerboard. This may be a bit too big of a change to do before the 2.80 release though, I don't personally have insight into that.

source/blender/makesrna/intern/rna_camera.c
234	I'm not sure why -10..10 limits were added here. If the intent was only to get a better step size and precision then I think "-FLT_MAX" and "FLT_MAX" would be better as soft limits.

Fixed matrices: currently a bit dump as every step in the evaluation is done as a separate matrix.

Harbormaster completed remote builds in B3901: Diff 15980.Jun 20 2019, 4:14 PM

Fine as it is.

Ideally it would probably first draw with the render engine with proper alpha channel, then alpha under the background images, and then alpha under the checkerboard. This may be a bit too big of a change to do before the 2.80 release though, I don't personally have insight into that.

Exactly my though. Shouldn't be too hard in practice but this can go after this diff is committed.

source/blender/draw/modes/object_mode.c
1525	This is correct. BLEND_ALPHA is the old BLEND

This revision is now accepted and ready to land.Jun 20 2019, 11:04 PM

Closed by commit rBfed6c1a970f1: Fix T62876: Camera Background Images (authored by Jeroen Bakker (jbakker)). · Explain WhyJun 21 2019, 9:57 AM

This revision was automatically updated to reflect the committed changes.

Jeroen Bakker (jbakker) marked an inline comment as done.

Jeroen Bakker (jbakker) added a commit: rBfed6c1a970f1: Fix T62876: Camera Background Images.

Jeroen Bakker (jbakker) mentioned this in T65988: When rendering in viewport mode the draw engines do not pass correct Alpha values..Jun 21 2019, 10:44 AM

Sebastian Koenig (sebastian_k) mentioned this in T66777: Background images are obstructed by invisible geometry..Jul 12 2019, 4:50 PM

Camera Background Images
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 16008

source/blender/blenkernel/intern/movieclip.c

source/blender/blenloader/intern/readfile.c

source/blender/draw/CMakeLists.txt

source/blender/draw/engines/gpencil/gpencil_engine.c

source/blender/draw/engines/gpencil/shaders/gpencil_fill_frag.glsl

source/blender/draw/engines/gpencil/shaders/gpencil_point_frag.glsl

source/blender/draw/engines/gpencil/shaders/gpencil_stroke_frag.glsl

source/blender/draw/intern/draw_manager.c

source/blender/draw/modes/object_mode.c

source/blender/draw/modes/paint_texture_mode.c

source/blender/draw/modes/shaders/common_colormanagement_lib.glsl

source/blender/draw/modes/shaders/object_camera_image_frag.glsl

source/blender/draw/modes/shaders/object_camera_image_vert.glsl

source/blender/draw/modes/shaders/object_empty_image_frag.glsl

source/blender/draw/modes/shaders/paint_texture_frag.glsl

source/blender/editors/space_view3d/view3d_draw_legacy.c

source/blender/gpu/GPU_texture.h

source/blender/gpu/intern/gpu_draw.c

source/blender/makesdna/DNA_movieclip_types.h

source/blender/makesrna/intern/rna_camera.c

Camera Background ImagesClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 16008

source/blender/blenkernel/intern/movieclip.c

source/blender/blenloader/intern/readfile.c

source/blender/draw/CMakeLists.txt

source/blender/draw/engines/gpencil/gpencil_engine.c

source/blender/draw/engines/gpencil/shaders/gpencil_fill_frag.glsl

source/blender/draw/engines/gpencil/shaders/gpencil_point_frag.glsl

source/blender/draw/engines/gpencil/shaders/gpencil_stroke_frag.glsl

source/blender/draw/intern/draw_manager.c

source/blender/draw/modes/object_mode.c

source/blender/draw/modes/paint_texture_mode.c

source/blender/draw/modes/shaders/common_colormanagement_lib.glsl

source/blender/draw/modes/shaders/object_camera_image_frag.glsl

source/blender/draw/modes/shaders/object_camera_image_vert.glsl

source/blender/draw/modes/shaders/object_empty_image_frag.glsl

source/blender/draw/modes/shaders/paint_texture_frag.glsl

source/blender/editors/space_view3d/view3d_draw_legacy.c

source/blender/gpu/GPU_texture.h

source/blender/gpu/intern/gpu_draw.c

source/blender/makesdna/DNA_movieclip_types.h

source/blender/makesrna/intern/rna_camera.c

Camera Background Images
ClosedPublic
Actions

Revision Contents
Changeset List