CUDA 10.2 and GTX 1080 render artifacts with path tracing
Closed, ResolvedPublicBUG
Actions

Description

System Information
Operating system: Linux-5.3.0-7629-generic-x86_64-with-Ubuntu-19.10-eoan 64 Bits
Graphics card: GeForce RTX 2080 SUPER/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 440.44

Blender Version
Broken: version: 2.83 (sub 6), branch: master, commit date: 2020-03-04 04:24, hash: rBfb330dd11098
Worked: 2-3 days ago

Short description of error
One of the recent changes to Cycles has introduced a Path Tracing sampling error when using CUDA on a 1080.
It does not happen with an RTX 2080, and also not when using OptiX. It also does not occur when using BPT.

Exact steps for others to reproduce the error
Open this file and render.

cycles_cuda_bug.blend574 KBDownload

Revisions and Commits

rC Cycles
	rC480f9ef4dc65 Fix T74423: Cycles rendering artifacts with CUDA 10.2
rB Blender
	rBcf5147f69f0e Fix T74423: Cycles rendering artifacts with CUDA 10.2

Related Objects

Mentioned In: T74464: Adaptive Sampling Giving Weird Results on GPU
Mentioned Here: rB409074aae561: Cycles: add Progressive Multi-Jitter sampling pattern
rBcd0a6ff5c453: Fluid: Diffusion settings now optional

Event Timeline

Sebastian Koenig (sebastian_k) created this task.Mar 4 2020, 9:52 AM

I can't reproduce this with the rB3da2dc82132f build from builder.blender.org/download/, using Quadro RTX 5000.

I can't find a buildbot build with revision rBfb330dd11098, is this a custom build?

Can not reproduce with blender-2.83-31aefdeec5a6 build from yesterday.
Windows 10 Pro
Geforce 1080 Ti
Geforce GTX Titan X
Driver: 440.97

Hm, that's weird. I just did a fresh build (cd0a6ff5c453), and still have the issue. I have 3 cards, two 1080 and one 2080. I can test with both GTX cards and the issue is always there, just not with the RTX. I also have no patches or so applied.

Can you test with the buildbot build, so we know if the difference is in the build or the hardware/drivers?

Brecht Van Lommel (brecht) mentioned this in T74464: Adaptive Sampling Giving Weird Results on GPU.Mar 5 2020, 2:15 PM

Might indeed be the 10.2 toolkit, as @Steffen Dünner (SteffenD) mentioned in the other report. I just tried with buildbot and that one renders without issues.

Does not reproduce for me either with RTX Q4000, CUDA V10.2.89, gcc version 7.4.0, Nvidia drivers 440.33.01.

It's probably not CUDA 10.2 itself but something else.

@Sebastian Koenig (sebastian_k) since you're making your own builds, do you have the time to git bisect the commit that introduced the bug?

It's not unusual for new CUDA toolkit versions to give us bugs like this, and us having to fix or work around it. Sometimes it's a bug in Cycles, sometimes it's a bug in CUDA.

git bisect may reveal the issue, but to me it seems unlikely to have been a recent commit.

Brecht Van Lommel (brecht) renamed this task from CUDA sampling bug with Path Tracing to CUDA 10.2 and GTX 1080 render artifacts with path tracing.Mar 5 2020, 2:49 PM

Brecht Van Lommel (brecht) merged a task: T74464: Adaptive Sampling Giving Weird Results on GPU.

In T74423#885752, @Sebastian Koenig (sebastian_k) wrote:

Might indeed be the 10.2 toolkit, as @Steffen Dünner (SteffenD) mentioned in the other report. I just tried with buildbot and that one renders without issues.

It looks like this is the solution. In my case removing CUDA 10.2 and installing 10.1 got rid of the artefacts.

Marcus Papathoma (machieb) removed a subscriber: Marcus Papathoma (machieb).Mar 5 2020, 2:51 PM

git bisect may reveal the issue, but to me it seems unlikely to have been a recent commit.

If 10.1 does get rid of the error, than I also think it is not a recent commit. I did install 10.2 recently, shortly before @Stefan Werner (swerner) committed the progressive multi-jitter. If thought it was because of that, but yeah, it seems more likely that the new cuda toolkit is indeed the culprit.

I plugged in my GTX 1080Ti and git blame points to the commit that brought the PMJ sampler:
https://developer.blender.org/rB409074aae56138f49ce078ce919a6d02e44e521e

It reproduces with the scene attached to this bug, regardless of the sampling pattern that is selected.

Stefan Werner (swerner) changed the task status from Needs Information from Developers to Confirmed.Mar 5 2020, 4:39 PM

Hmmm... I can not reproduce this error on Linux Mint 19.3 with a GTX 1080Ti.
Both the current buildbot build and my own build (with CUDA 10.1) render properly:

I reported a bug with adaptive sampling before, but that was simply because I compiled using CUDA 10.2
Using CUDA 10.1 fixed the adaptive sampling, what version of CUDA are you building with?

Brecht Van Lommel (brecht) reopened this task as Confirmed.Mar 5 2020, 6:18 PM

Brecht Van Lommel (brecht) closed this task as a duplicate of T74471: Cycles GPU artifacts.

Brecht Van Lommel (brecht) merged a task: T74471: Cycles GPU artifacts.

Brecht Van Lommel (brecht) added a subscriber: Andrew Enyart (arenyart).

CUDA 10.2 + PMJ commit shows the render error. CUDA 10.2 and any revision before the PMJ commit renders fine.

Reproduces with 1080Ti/CUDA 10.2 on Windows as well.

@Stefan Werner (swerner) is this something you plan to fix? I don't see anything obviously wrong in the code, so I guess it's a commenting out code until the cause is found.

If it helps and this is caused by a compiler bug, we could also consider removing CMJ sampling.

Oliver Weissbarth (oweissbarth) added a subscriber: Oliver Weissbarth (oweissbarth).Apr 20 2020, 11:38 AM

Brecht Van Lommel (brecht) closed this task as Resolved by committing rBcf5147f69f0e: Fix T74423: Cycles rendering artifacts with CUDA 10.2.Apr 22 2020, 4:42 PM

Brecht Van Lommel (brecht) claimed this task.

Brecht Van Lommel (brecht) added a commit: rBcf5147f69f0e: Fix T74423: Cycles rendering artifacts with CUDA 10.2.

Brecht Van Lommel (brecht) merged a task: T75967: visual artifacts with GPU but not CPU using cycles.Apr 22 2020, 4:43 PM

Brecht Van Lommel (brecht) added subscribers: Ed Beroset (beroset), Midge Sinnaeve (mantissa).

Brecht Van Lommel (brecht) added a commit: rC480f9ef4dc65: Fix T74423: Cycles rendering artifacts with CUDA 10.2.Jun 2 2020, 7:47 AM