Some results comparing GTX 1080 and Core i7-4790K + GTX 1080, on Linux with tile size 32x32.

Some scenes are scaling ok, others not so much. It would help to support rendering smaller tiles faster on the GPU or to let multiple CPU cores work on the same tile. These are relatively short 10-25s renders though, I expect it would be less of an issue with more samples.

Reserving a thread for CUDA may be needed, or perhaps raising its priority, though disabling the spin loop seems not too bad here. On Windows the OS thread scheduler may not work so well though.

replacing line 1374 with:

if entry.type == 'OPENCL' or entry.type == 'CPU':
    if entry not in cuda_devices:
        opencl_devices.append(entry)

makes OpenCL, CUDA and CPU to work together, however, with OpenCL having a bottleneck under 64x64 tiles, it is slower to render with CPU.

Supporting CUDA + OpenCL rendering would be nice indeed. Mainly the challenge there is changing the UI so we only get a single device list instead of CUDA and OpenCL separately, and the associated changes to preserve backwards compatibility.

Regarding performance, there's a bunch of factors:

• If lowering the number of CPU threads in the Performance panel to "number of CPU cores - number of GPUs" helps, that means the GPU thread will need higher priority or we will need to do such a heuristic automatically.
• The addition of CU_CTX_SCHED_BLOCKING_SYNC may slow down GPU rendering (with or without adding the CPU for rendering), this could be left out but requires lowering the number of CPU threads.
• If the tile is too big it can happen that the GPU finishes early and then has to wait on CPU threads to finish their tiles.

In D2873#67419, @Brecht Van Lommel (brecht) wrote:

• If the tile is too big it can happen that the GPU finishes early and then has to wait on CPU threads to finish their tiles.

This could be a good time to implement different tile size for GPU/CPU, or if it´s possible, an adaptive one that subdivides the remaining threads according to the number of devices and work left.

Just an idea, I´m not sure if this can be done or if it´s possible in the current development.

The plan is to let each GPU render many small tiles at once. Different tile sizes or subdivision would still constrain the tile shape to be rectangular which doesn't always fit well.

from test made in UI, cpu indeed is always the last one to finish. The more threads a cpu will have, the higher the probability is that the GPUs will idle, because those 16 or 32 tiles are already being rendered by cpu very slowly. So if it's possible without too much work, I would say it would be more effective in real scenarios to let all the CPU thread render one tile, just like all the thread of the GPU render one tile. It may also improve cache behaviour and increase render speed. Of course, letting GPU render several tiles would still be needed to ensure better occupancy.

Viewport rendering works quite poorly with this, since it basically splits the viewport in half between the CPU and GPU. If the GPU is e.g. 4x faster than the CPU this slows things down overall. Perhaps the CPU should remain disabled for viewport render, or only kick in after a few seconds if we can figure out a way to do better work distribution then with bigger numbers of samples at a time.

viewport rendering of BMW from official benchmark pack takes 12seconds on 1080TI, 20seconds on Vega64 and 16 seconds using both. With F12 render, that's the opposite, Vega is faster with 82sec (at 128x128, best time), 1080Ti takes 93seconds (at 16x16, best time) and both take 44seconds using latest master with initial_num_samples at 5000.
To sum up:

• viewport seem really slow in latest master. OpenCL. 2.78c with selective node compilation for viewport renders nearly 2x faster on Vega 64. It's not due to SSS or volume as those are not compiled in viewport kernel either. I can investigate on that.
• multi-device rendering is slower with viewport/progressive rendering than the fastest device alone. Logic would be to wait for the slowest half to finish, which would be around 10seconds for Vega?

In D2873#67613, @mathieu menuet (bliblubli) wrote:

• viewport seem really slow in latest master. OpenCL. 2.78c with selective node compilation for viewport renders nearly 2x faster on Vega 64. It's not due to SSS or volume as those are not compiled in viewport kernel either. I can investigate on that.

I guess the bicubic stuff is part of that, but we should indeed figure out what the other reasons are.

• multi-device rendering is slower with viewport/progressive rendering than the fastest device alone. Logic would be to wait for the slowest half to finish, which would be around 10seconds for Vega?

It's not that simple probably. For example a few paths with many bounces may take some minimum amount of time to be finished (especially with a split kernel), and the only way to hide that may be to render more samples at a time the way we do in batch.

In general scheduling of interactive rendering with different GPUs is a very difficult problem. We could do a little better than splitting half by for example using the number of cores multiplied by the clock speed to find a more appropriate split, but between NVidia and AMD it's also not that meaningful. And probably would not solve this particularly latency problem.

is there possibility to use all cpu cores to render single rectangle at one time?

Not at the moment, but maybe in the future.

In D2873#67481, @Brecht Van Lommel (brecht) wrote:

The plan is to let each GPU render many small tiles at once. Different tile sizes or subdivision would still constrain the tile shape to be rectangular which doesn't always fit well.

Is there currently an actual to do for this task? Machines with multiple decent GPUs are almost always worse to enable CPU+GPU than just doing GPU alone, I have a 1950x, 2 GTX 1070s and a RTX 2080. Enabling my 32 thread CPU rarely speeds up anything because I need to have 16x16 tiles to effectively render CPU but about 70% of my power is in GPU so limiting it to 16x16 is crippling my GPUs just so the CPU doesn't take DAYS on larger tiles