Page MenuHome

Compositor: Add support for canvas compositing
ClosedPublic

Authored by Manuel Castilla (manzanilla) on Sep 12 2021, 8:23 PM.
Tokens
"Like" token, awarded by GeorgiaPacific."Burninate" token, awarded by gilberto_rodrigues."Love" token, awarded by Alaska."Love" token, awarded by EAW.

Details

Summary

This patch adds functionality for operations that require pixel
translation or resizing on "Full Frame" mode, allowing to adjust
their canvas. It fixes most cropping issues in translate, scale,
rotate and transform nodes by adjusting their canvas to the result,
instead of the input canvas.

Operations output buffer is still always on (0,0) position for
easier image algorithm implementation, even when the
canvas is not.

Current limitations (will be addressed on bcon2):

  • Displayed translation in Viewer node is limited to 6000px.
  • When scaling up the canvas size is limited to the scene resolution size x 1.5 . From that point it crops.

If none of these limitations are hit, the Viewer node displays
the full input with any translation.


Test file:

BeforeAfter

Diff Detail

Repository
rB Blender
Branch
arcpatch-D12466 (branched from master)
Build Status
Buildable 17370
Build 17370: arc lint + arc unit

Event Timeline

Manuel Castilla (manzanilla) requested review of this revision.Sep 12 2021, 8:23 PM
Manuel Castilla (manzanilla) created this revision.

Seems to ignore translate and rotate would use the buffer resolution. I would think that lens distortion always works in screen space.

There are always some exceptions in nodes that requires to work in screen space (apply the canvas, before executing the node).

  • Lens Distortion
  • Sun flare
  • Glare node
  • Movie distortion
  • Crop
  • ...

Or should D12465 be first applied?

  • Use resolution center as rotation center (without canvas offset)
  • Make lens distortion work in screen space

Need to confirm "translate + rotate" and lens distortion node are ok now, before doing it for all the nodes.
Apart from the nodes you said, would need to add only: Plane Track Deform, Vector Blur and Stabilize 2D?

  • Fix last diff submitted from wrong branch

Usability:

  • When attaching a viewer node to a node it should show it in canvas space. Eg. image -> translate 100 in x -> viewer should show the image translated 100px. Currently it shows the image as the translation didn't happen. Rotate node seems to do something similar.
  • Rotate node seems to be incorrect.

source/blender/compositor/intern/COM_CompositorContext.h
291

Use BLI_float2.hh as return type.

We could add a using to have a more descriptive name (Size2f).

float2 get_render_size()

  • Use float2 for render size
  • Show translation in viewer node within an unlimited canvas
Manuel Castilla (manzanilla) marked an inline comment as done.Sep 15 2021, 2:23 AM

When attaching a viewer node to a node it should show it in canvas space. Eg. image -> translate 100 in x -> viewer should show the image translated 100px.

I've done this, but more translation means a bigger viewer buffer and it affects performance. I guess I have to do it with buffer paddings if we want to show it in image editor and not overwrite UI backdrop offset.

The rotate issue I see in the video is because of backdrop late refresh of the resolution. It happens with any node that dynamically changes its canvas size, the viewer uses that size. I can't come up with any solution other than displaying the input within the scene output canvas, same as composite node.

Looking at it in more detail, the viewer node is updates the image editor, even when the compositor is canceled, but working on incomplete buffers. I think the fix for now is that for full frame compositor we should not update the image editor/backdrop as it gives incorrect data to the user (better to give previous correct state than new incorrect state).

In order to add this patch for bcon1 I would suggest to make sure that the backdrop only displays correct data. If the basic transform operations work we can add it to master. We can during bcon2 do the in depth testing and fine tuning. based on user feedback.

Jeroen Bakker (jbakker) requested changes to this revision.Sep 20 2021, 9:21 AM
This revision now requires changes to proceed.Sep 20 2021, 9:21 AM
  • Minimize display buffer updates on cancelling
  • Scale from resolution center to match tiled implementation
  • Use scale, rotate and translate operations in Transform node
  • Use scale, rotate and translate operations in Stabilize2D node
  • Do not clear display buffer on cancelling
  • Use intptr_t for offsets to avoid overflows when scaling up
  • Increase max scale canvas size to allow some translation in output canvas without cropping
  • Limit viewer translation padding
  • Cleanup: unused variables

Or should D12465 be first applied?

It's ok just applying this patch, the other is the base of this.

All basic transform operations should now work the same way as tiled implementation.
Transform nodes are now using Scale, Rotate and Translate linking instead of TransformOperation for easier maintenance, until
it's well defined how the basic transform operations should work.
I've limited displayed translation to 6000px to avoid hangs until we implement the backdrop offset for the viewer node.

It seems like we are really close. I detected some pixel differences between tiled and full frame rotation but that might just be an acceptable difference due to the buffering.


There is a crop difference in the next scene between Tiled and Full frame compositing:

I think it is doable to get this in master this week. I would still put this under experimental feature and get initial feedback from actual compositors, before removing the experimental flag.
In Bcon2 we should focus on cleanup (also code) and fixing issues. There might be some hidden regressions that can emerge.

Great work, didn't expect this to be a target for 3.0!

The file from T57568: Transform Node Messes Up Alpha Channel Manipulation is less cropped than with Tiled using the Transform node, but there is still some cropping.

Using a Scale and Rotation node in either order doesn't have this issue as they make the canvas larger.

Not realizing that the Scale node X and Y inputs are factors of image size instead of pixels leads to some interesting results. =)

EDIT:
I tested and submitted this before I saw the posts by @Jeroen Bakker (jbakker) that cover the same thing.

  • Fix previews cropping inputs
  • Fix texture node having different canvas in full frame

The cropping issues are due to the scale canvas limitation (scene output resolution * 1.5). We've agreed to look for alternatives to this on bcon2.
@Evan Wilson (EAW) Thank you for the feedback, I've reproduced the third case and was a problem with the texture node. It should be fixed now.
If there is still differences, can you share the file please?

This revision was not accepted when it landed; it landed in state Needs Review.Sep 28 2021, 10:02 PM
This revision was automatically updated to reflect the committed changes.

Committed on agreement with the reviewer.