Paths

Table of Contentst

Cycles: Experiment with native cubemap support in Cycles
Needs ReviewPublic
Actions

Authored by Sergey Sharybin (sergey) on Mar 26 2016, 5:57 PM.

Details

Reviewers

Brecht Van Lommel (brecht)
Martijn Berger (juicyfruit)

Summary

This commit implements a native cubemap rendering in Cycles,
which is currently a research project with the following goals:

Make it easier to render cubemaps for engines which supports or expects them to be used.

Intermediate projection to optimize rendering of equirectangular camera.

Cube map layout is not currently standart but it's most efficient
from the resolution and storage point of views and it's currently:

+-------+-------+--------+
| Back  | Right | Bottom |
+-------+-------+--------+
| Front | Left  |  Top   |
+-------+-------+--------+

It's easy to re-map views in the future.

There are still issues to be solved, but it'll be nice to get
VR and panorama artists involved into feedback already. So the
limitations are:

Anti-aliasing is not perfect currently. It feels to be be rather correct for rendering, but using it as an environment map will cause unexpected color bleed on the side edges.

There's no auto-remapping to equirectangular map happening yet, so the only way to see result is to put result to an environment map with cubemap projection and either re-render or investigate it in viewport.

Projection formulas are not optimized by any mean. This is to keep them totally obvious to make it easier to make further tweaks.

Diff Detail

Repository: rB Blender
Branch: cycles_panorama_cubemap

Event Timeline

Sergey Sharybin (sergey) updated this revision to Diff 6320.Mar 26 2016, 5:57 PM

Sergey Sharybin (sergey) retitled this revision from to Cycles: Experiment with native cubemap support in Cycles.

Sergey Sharybin (sergey) added reviewers: Martijn Berger (juicyfruit), Brecht Van Lommel (brecht).

Sergey Sharybin (sergey) updated this object.

Sergey Sharybin (sergey) added a subscriber: Sebastian Koenig (sebastian_k).

Just to stress it again: it's more a re-search project, more asking for a workflow type of comments, issues with filtering will be worked on soon (they are known).

However, maybe someone will be able to find a nicer solution for direction_to_cubemap(). Perhaps we only need D and side of a cube where to project on will be denoted by a maximum component of this vector?

Eeeh, forgot to attach example files again. Sorry for the spam.

Scene which is set up to render cubemap:

cornell_box_pano_cubemap.blend229 KBDownload

Scene which is set up to view cubemap:

cubemap.blend436 KBDownload

Cubemap example:

Brecht Van Lommel (brecht) added inline comments.Mar 26 2016, 7:17 PM

intern/cycles/kernel/kernel_projection.h
214	This is equivalent to `if (N.x > N.y && N.x > N.z)` or `if (den == abs_N.x)`.
238	Best make this just `else {`, in case some float precision issues lead to uninitialized `u` and `v`.

There's a pretty clever way to get seamless texture filtering, by modifying the mapping in both directions so that there ends up being a 1 pixel border replicating the pixels from neighbouring faces.
http://the-witness.net/news/2012/02/seamless-cube-map-filtering/

For bicubic lookups you'd need to make it a 2 pixel border. The advantage of this is that all the logic can be contained in the mapping functions, the disadvantage is that the resulting cubemaps are not quite standard then, though I'm not sure we care much about that here.

@Brecht Van Lommel (brecht), Thanks for the link. This is something i was gonna to test and see how it works, but now i see it will indeed work :)

So now only mystery is making it really optimal and easy to use workflow, which i hope to catch @Sebastian Koenig (sebastian_k) in irc to extract some knowledge from him.

intern/cycles/kernel/kernel_projection.h
238	That's a good idea.

Some improvements and fixes:

Antialiasing should work fine now

It uses border of 2 pixels, which requires having raster resolution in projection, which caused some chain changes to feed those funcitons with required data.

Simplified cube projection

Still to go:

OSL support

Support of more industry standard view layout (or we do it as post-process, so we wouldn't care about view layout in kernel at all?)

Auto-remapping to equirectangular (perhaps?)

I tried it (before the most recent update) and so far it seems to work good, though I couldnt test it in VR yet. Having an option to use different mappings would be great, especially the most common one where you have a long stripe of images. I don't want to use the term industry standard, because one, the way that mapping is set up is super stupid (in my opinion) and things change anyway all the time in VR world... :)
Anyway, awesome to see this progress.
What we use in the studio all the time is Dalai's cube map add-on: https://github.com/dfelinto/render_cube_map
It's a bit of a hacky solution, but the nice thing is that you can render the different sides separately, which I think would be a nice addition to the "official" version too: you can distribute the single tiles to multiple machines, compositing works better and you can render single tiles again to fix mistakes.
When I am back in office I will test a bit more.

@Sebastian Koenig (sebastian_k), surely delivering standard views layout is something we have to implement. What i'm not real fan of doing is making kernel itself aware of all those combinations..

Thinking of post-pro type of re-mapping, so same cubemap from cycles can be converted to either equirectangular or cubemap with different views arrangement. It'll stress render pipeline in blender a bit (which currently expects same resolution all over the pipeline) but sounds closer to what artists actually expects to happen.

Surely re-mapping can happen as a separate re-render, but that'd be cool to avoid.

As for separate views render with a cube map: again, not really looking forward of adding such complexities to a render pipeline itself. This could be achieved by using render border. We can have an addon to help setting a render border for local renders. For renderfarms it's something what renderfarm software can control.

Actually, if it's just to speed up render of a single frame, then it should not matter for render farm if it's a cube map or equirectangular or just regular perspective projection: farm will split frame based on tiles, and it's not forbidden to have different views in a tile.

Btw, less intrusive idea (from pipeline.c point of view) would be to have an operator or API call which either creates new image datablock or arranges pixels in a current one to reflect whatever mapping you want. So you can run "Convert Cubemap to Equirectangular" operator or something like that.

P.S. Perhaps term "view" wasn't great in the comment (because of stereo multiview), read it more as a "cube side".

@Sebastian Koenig (sebastian_k), one more thing. I've went ahead and merged both cubemap and lambert patches into a single branch called cycles_panorama_experiments. So now it should be easy for you to test both projections, compare them and always have latest fixed versions.

mathieu menuet (bliblubli) added a subscriber: mathieu menuet (bliblubli).Jul 10 2016, 1:51 PM

Dear Brecht, I have been experimenting with generating and using cubemaps in Blender in SVM, OSL, and GLSL, to later find this old thread. I encountered the same issues with (bicubic) filtering and used a different strategy: force nearest-neighbor near the cube edges. My solution avoided the filtering issue at higher mipmap levels (e.g.: during fast EEVEE interactivity) (and without the trick from The Witness), but doesn't cover 'baking' (afaik). I don't mind that my code changes overlap with this current thread (mine was just an exercise in coding), but mainly wonder if the cycles_panorama_experiments branch (and the work in the current 2-year-old thread) is still of interest to current Blender. And whether I should try adding to Sergey's work? I understand the current workload you have with 2.80, so postponing is not a problem :). Then again, maybe, camera mapping is something to be solved with "everything nodes"?

Brecht Van Lommel (brecht) added projects: Render & Cycles, Cycles.Sep 22 2020, 1:42 PM