Page MenuHome

Cycles-X: Change viewport update system to be time based rather than sample based on fast devices
ClosedPublic

Authored by Alaska (Alaska) on Aug 19 2021, 7:53 AM.
Tags
None
Subscribers
None
Tokens
"Like" token, awarded by hitrpr."Love" token, awarded by Raimund58."Love" token, awarded by gilberto_rodrigues."Love" token, awarded by HEYPictures."Love" token, awarded by franMarz.

Details

Summary

The viewport update speed when rendering with infinite samples has been changed to work based primarily on time since render start rather than sample count.

This change has been made because if you have a device fast enough to exceed 32 samples in 0.1 seconds, then the next viewport update you'll observe is 2 seconds after render start which can be annoying for anyone wishing to quickly iterate on fine details as render start is usually a 1 sample per pixel image meaning fine detail is lost in noise or overly blurred by a denoiser.

This change was made possible with help from Will Rusthon (a friend) and Blender developers.

Here is an example video of the issue (some of the "viewport changes" you see are due to compression artifacts to keep the video small):

And here is my fix tested (the second demo with the caustics is there to show that the viewport updates reduce as time goes on. It's easier to see in the caustics scene due to the fireflies. This is important because reduced viewpoint updates means faster ray tracing. The difference is that with this patch the slow viewport update (once every 2 seconds) has been moved to after rendering for 8 seconds rather than occurring at 32 samples which could happen really quickly with simplistic scenes. Also the denoiser stopping at the end of the video has been fixed and is currently in the patch available for review):

Diff Detail

Repository
rB Blender
Branch
cycles-x-time-based-viewport-updates (branched from master)
Build Status
Buildable 16519
Build 16519: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Fix compile error

Fix issue with denoiser not functioning after a short while

intern/cycles/integrator/render_scheduler.cpp
813–814

I'm not 100% sure if

(time_dt() - state_.start_render_time) > 4
or
(time_dt() - state_.start_render_time) > 8

should be here.

Alaska (Alaska) edited the summary of this revision. (Show Details)Aug 19 2021, 8:40 AM
Alaska (Alaska) edited the summary of this revision. (Show Details)

Remove unnecessary TODO

Alaska (Alaska) marked an inline comment as not done.Aug 19 2021, 8:50 AM
Alaska (Alaska) added a comment.EditedAug 19 2021, 9:21 AM

There is the ability to add a fall back to the old behavior by replacing things like

if ((time_dt() - state_.start_render_time) < 1) {

with

if (((time_dt() - state_.start_render_time) < 1) || num_rendered_samples < 4) {

This fall back to the old behavior will result in faster viewport updates for scenes that render slowly.

For example, if a scene take 8 seconds to reach 5 samples, the fall back will result in faster viewport updates (in the specific scenario described, the viewport could update 1.75 seconds faster).


A fall back for the denoiser would also need to be added?
Something like this maybe? (I'm not sure if some of these brackets are necessary)

delayed = ((((time_dt() - state_.start_render_time) > 4) && (num_samples_finished >= 20)) && (time_dt() - state_.last_display_update_time) < 1.0);
Alaska (Alaska) added a comment.EditedAug 19 2021, 9:26 AM

Also, adaptive sampling is unaffected. I'm not sure how to change this.

Add fall back to old method in cases that it is faster

Sergey Sharybin (sergey) requested changes to this revision.Aug 19 2021, 10:10 AM

Think using time instead of samples is better idea and could be a good low hanging fruit to grab.

Ideally this would somehow be based on actual variance of the image, but currently we don't have an easily accessible mechanism for this (especially when adaptive sampling is not used). So to me this ideal solution is something to look into in the future.

intern/cycles/integrator/render_scheduler.cpp
635

Avoid subsequent calls of time_dt():

const double render_time = time_dt() - state_.start_render_time;

if (render_time < 1) {
  ...
}
813–814

Similar to above: avoid subsequent time_dt():

const double current_time = time_dt();
delayed = ((current_time - state_.start_render_time) > 4 && (current_time - state_.last_display_update_time) < 1.0);

Think for now starting delaying sooner is better.

This revision now requires changes to proceed.Aug 19 2021, 10:10 AM

Make changes to match comments by Sergey

Alaska (Alaska) marked 2 inline comments as done.Aug 19 2021, 11:15 AM

Fix extra bracket

Fix another bracket

Remove more unnecessary brackets

Fix unnecessary changes to whitespace

Alaska (Alaska) edited the summary of this revision. (Show Details)Aug 19 2021, 1:26 PM

Update comments

Alaska (Alaska) edited the summary of this revision. (Show Details)Aug 20 2021, 11:33 AM
Alaska (Alaska) edited the summary of this revision. (Show Details)Aug 20 2021, 11:43 AM
Alaska (Alaska) edited the summary of this revision. (Show Details)

Update comment

Alaska (Alaska) retitled this revision from Cycles-X: Change viewport update system to be time based rather than sample based to Cycles-X: Change viewport update system to be time based rather than sample based on fast devices.Aug 21 2021, 8:06 AM

Make adjustments to the measurement of time to be time rendering rather than time since render start as it can cause some issues with renders that can be paused. Also made small adjustments to clean up code.
Done with the help of @William Leeson (leesonw)

Update comment and fix faster display updates at lower pixel sizes

intern/cycles/integrator/render_scheduler.cpp
653

I have replaced 1.0 with pixel_size_ * pixel_size_ because other wise it guesses that the device is significantly faster than it actually is when the pixel_size_ is larger than 1. This lead to significantly faster display updates when using a larger pixel sizes which leads to a loss of performance.

pixel_size_ * pixel_size_ is used because although the value might be 8x, the actual resolution drop/performance increase is 8*8.

I have a few questions related to this.

Should this change be made? Having viewport updates being consistent between pixel sizes is useful because it's consistent. However, with lower resolutions, faster viewport updates may be preferred as more perceptual change can occur in the same time span when compared to a higher resolution.

Also, should this change be made here? The reason this change was made is because path_trace_time_.get_average(); is incorrect by the multiple of the pixel_size_ squared. Should I change that value to be correct rather than changing it here. The issue with this is that it might lead to issues else where. Because this is what it reads at line 346 onwards in render_scheduler.h.

/* Timing of tasks which were performed at the very first render work at 100% of the
 * resolution. This timing information is used to estimate resolution divider for fats
 * navigation. */
struct {
  double path_trace_per_sample;
  double denoise_time;
  double display_update_time;
} first_render_time_;

TimeWithAverage path_trace_time_;
TimeWithAverage adaptive_filter_time_;
TimeWithAverage denoise_time_;
TimeWithAverage display_update_time_;
TimeWithAverage rebalance_time_;

I'm most focused on Timing of tasks which were performed at the very first render work at 100% of the. Does this apply to TimeWithAverage path_trace_time_; ? Is 100% resolution the viewport resolution or the viewport resolution after the pixel size modifier?

Alaska (Alaska) marked an inline comment as not done.Aug 23 2021, 1:14 PM

I've done some further investigation and realize some of what I said in the previous comment is incorrect. The change to accommodate pixel size into path_trace_time_ might have to be made in /intern/cycles/integrator/path_trace.cpp

However, accommodating pixel size in path_trace_time_ will lead to issues else where.

William Leeson (leesonw) requested changes to this revision.Aug 23 2021, 1:46 PM

This looks fine to me I don't think it is necessary to move the pixel_size into path_trace.cpp as this kind of stuff has more to do with the scheduler than the path tracer IMHO. However, I'll have to defer to @Brecht Van Lommel (brecht) or @Sergey Sharybin (sergey) to accept this.

This revision now requires changes to proceed.Aug 23 2021, 1:46 PM

Fix spelling in comment

Remove check for sample count during first 4 seconds

Alaska (Alaska) added a comment.EditedAug 24 2021, 9:47 AM
  if (render_time < 1 || num_rendered_samples < 4) {
    return 0.1;
  }
  if (render_time < 2 || num_rendered_samples < 8) {
    return 0.25;
  }
  if (render_time < 4 || num_rendered_samples < 16) {
    return 0.5;
  }
  if (render_time < 8 || num_rendered_samples < 32) {
    return 1.0;
  }
  return 2.0;
}

I've had a thought about this part, specifically the num_rendered_samples < Y. I implemented it based on an idea that turns out to not be true upon further investigation of the code.

As a result
render_time < 1 || num_rendered_samples < 4
render_time < 2 || num_rendered_samples < 8
render_time < 4 || num_rendered_samples < 16
are basically useless. render_time < 8 || num_rendered_samples < 32 on the other hand does still help but only in a little bit.

I had questions relating to this. Since the first three options are basically useless, they have been removed. But should I also remove the last option?
For the last option is only useful in a specific situation. When you're rendering less than 4 samples per second and more than roughly 1 sample per second (I've had trouble with doing the math on this and I'm not actually sure what the minimum value is). If you meet that criteria, then display updates can be faster up to the 32nd sample when compared to removing num_rendered_samples < 32. I believe this is good because getting quicker feedback during these low sample counts is important, but if that is justification for this, then I could move the num_rendered_samples < 32 check onto the render_time < 4 line.

Adjusting the values for render_time < X and num_rendered_samples < Y and the return Z values will also change how useful these values are. However this might be outside the scope of this patch. There is a TODO earlier within the file with regards to tweaking these values for the best results:

Line 604
TODO (sergey): This is just a quick implementation, exact values might need to be tweaked based on a more careful experiments with viewport rendering. */

This looks good to me. I tried it out with a few different setups. There is probably no perfect answer for

guess_display_update_interval_in_seconds_for_num_samples_no_limit

but it seems to work and make sense.

This revision is now accepted and ready to land.Aug 27 2021, 3:56 PM