Nuke has a free-as-in-beer Personal Learning Edition (PLE) available for download if someone wishes to test this further:

http://www.thefoundry.co.uk/pkg_downloads.aspx?ui=CBC2593A-2C9F-4EF9-84BE-C198B0171453

On the slowdown, does it happen with normal (non-multilayer) openexr files?

If not, I suspect it's more of a side effect of using tiled EXRs. EXR files can be saved in various ways, using scanlines (with top->bottom or vice versa order defined by exporting application) or using tiles. With tiles, the openEXR library doesn't let you change the scanline order (well it seems to, but that option confusingly does something different).

The advantage of using tiles is that blocks can be written independently to the EXR file, rather than needing the entire image buffer. This is why it's been used for blender's multilayer files, since these are also used for things like 'save buffers'. I suspect in this case the tiled access may be slower than the scanline access, which is really something to do with the openEXR lib, not Blender.

However, I'll have a quick look to see if the multilayer render saving (i.e. not the save buffers feature) can just use scanline EXRs.. if so it will probably solve this, but I don't know how easy that will be.

Another note: Even with scanlines, the more channels, the slower to load the EXR will be. See: http://lists.gnu.org/archive/html/openexr-devel/2009-09/msg00008.html

Ok, I've done quite some extensive testing on this, here are the results:

* I was slightly incorrect above. Blender saves EXR files in 3 different ways:
- Rendering/saving to RGBA EXRs --> Scanline EXR
- Rendering/saving to multilayer EXRs --> Scanline EXR
- Saving temporary render results (save buffers option) --> Tiled EXR

So, tiled EXRs are only used for save buffers, therefore this is not a cause of any slowdowns for rendered multilayer EXRs

* In my tests, Blender's multilayer scanline EXRs are no slower than Blender's RGBA EXRs. I tested in both DJV and Nuke, and the multilayer file was only ~10% slower than the RGBA file.

This is consistent with how EXR works - EXR channels are saved interleaved, meaning that in order to retrieve channel data, the entire EXR file must be read. This means that file reading time scales close to linearly with the amount of information that's in it. By default Blender's multilayer EXRs contain an extra Z channel, which accounts for the 10% extra slowdown.

So if you're reading multilayer EXRs with lots of channels, it will be slower, at a rate proportional to the amount of channels in the file. This is part of EXR and has nothing to do with Blender.

So I'm happy to conclude there is no performance issue with EXR files saved out of Blender specifically.

* As for the upside down issue:
This is a bug in Blender - the OpenEXR specification dictates a display/data window with coordinate system starting with 0,0 in the top left corner of the image. Blender internally uses 0,0 at the bottom left.

"Pixel space is a 2D coordinate system with x increasing from left to right and y increasing from top to bottom. Pixels are data samples, taken at integer coordinate locations in pixel space." -- from OpenEXR TechnicalIntroduction.pdf

In Blender, there are two separate code paths/APIs for saving EXR files
- One used for saving RGBA EXR files
- One used for saving multilayer EXR files

It looks like the second API was added to be more general/flexible to support the sort of things needed for multilayer/tiled files, but whoever coded it probably never got around to replacing the first.

The first API handles correction for the different coordinate systems - using the right combination of starting point and 'y stride' it writes the EXR out in the expected order. However the second (multilayer) API has no correction for this, and therefore saves out invalid OpenEXR files.

I've attached a patch that mostly fixes this - it fixes reading both kinds of multilayer files, writing scanline multilayer files, and writing tiled multilayer files (almost). There's one small problem with the tiled multilayer files being offset by 1 pixel in Y that I wasn't able to exactly figure out yet - something to do with the render part 'crop' amount...

Also in a final version to commit to blender, it would be good to raise the Blender version saved in the openEXR header, and put in a switch to interpret exrs saved from old blender versions in the old way, and new exrs the correct way.

I hope someone else can help finishing up with this patch, I've spent quite a long time on it already!

I found the origin of the last bug:
For tiles in the upper part of the image we don't want to save all the lines if image height is not divisible by ypart.

While I can easily correct the code to position the rect ponter to the effectively last util line (and not the last line) for those tiles; another pronlem is raised.

When writing the tileto file, as the start line is not the last of our buffer, then the last written line is not the first line of our buffer neither (so we save pixels of the crop area).

I think the only ways around this could be:
- If ther is an option for erx files to have the not completely usefile tiles on the top of the image instead of the bottom

- If not then, modiffy the render code to have the tiles aligned in the upper left of the image.

Will continue investigating this, but if somebody with good openexr kmowledge could point to somme related doc, I would be really greatfull.

Xavier

Found somere lated stuff:

-First another bug: the tiles must be numeroted from 0 to nrtiles-1 which was the case for the xcoord but not the y. Somehow it still worked because the pixels coordinate for the tile offset were right. But sometimes (whe the image size is exactly divisible part number of parts) some tiles where not saved (wrong tile coord) which make blender crash when reading the files and prevent opening the file in other app.

- With that corrected and the first errors I found -> it works. I tryed comparing rendere result with and without save buffer and they are the identical (exept the tiled save buffer have empty rgba channels and populated RenderLayer.Combined channels while rendering wihtout save buffer and saving result have both populated but not in the same colorspace ...)

- Problem is that upper left pixel of upper left tile is not allign to openexr datawindow Xmin Ymin and the openEXR spec says it must be :(

- Don't know wy it still work (wihtout offest) but it works

- There is a way to define a display window to prevent this kind of problem, so i am on it...

to continue ...

Ok, I managed to saved all the image area covered by the tiles, align these tiles to upper left to conform with exr spec and defined a display area that cover only the render area.

I slightly modified the loading code to be able to reload such files and flip files saved with previous blender version.

If somebody can test the patch, please give me feedback (specially on windows as it is untested on this os)

Fix is in SVN, yay!

Reopened as svn commit was reverted because of thread issues causing crash :(
I included a simple file reproducing the crash (you may need to render several times before the crash produce) and a up to date patch with some corrections.

I investigated it quite a long time and here is what I found:

-OpenExr library must be initialized to be thread safe (done in new patch)
-There was a small error in the data window coordinates (corrected in new patch)
-The error is similar to what we have if we calculated wrong x/ystride or slice base pointer (sigsev in the same part of openexr library code)
-When the error occurs the y tile coord is always 0 (in exr coord so it is a tile of the upper side of the image)
-I tried runing with hellgrind, there is no warning regarding openexr but a lot of possible data races in the blender renderpart data.

Is it possible that the rendered tile is freed before being completely written? I seems not but maybe someone more familiar with render code can check.

I also added some lines of code in IMB_exrtile_write_channels() to handle buffer access with global coordinates (like before) instead of coordinates relative to the tile. You can
switch from one method to another by commenting/decommenting the code.

Maybe someone will have more luck than me with this.

Assigning to Brecht. I saw you corrected some render multi-threading issue recently which seems similar/related.

Patch openexr_coordsys04.txt still apply cleanly against recent svn.

Assign to self, will put on my todo.

I need to have verified what the status of our current code is. We have two formats for saving:

1) Regular OpenEXR RGBA with optional Z
2) Blender Multilayer OpenEXR

Both save out scanline files, and use shared code, so should not be upside down?

The tiled temp buffers during render are not meant to be shared with other applications, for that I'm not going to mess with the code now, I like to keep this local.

fix is in svn.

(Temp buffers use we keep local for a while)