This implements T56162, to add a fast alternative compression for blend file io.
- LZ4 (latest stable from https://lz4.github.io/lz4) is included in extern/ or the system LZ4 can be used.
- Compression has been changed from a boolean to an enum.
Initial Timing Tests
Tests with a file containing subdivided cubes (23 cubes, subdivided 8 times ~ 293216 faces), reading writing to a HDD (dropping caches to test for cold reads).
| Compression | Read (cold) | Read (warm) | Write | Size |
|---|---|---|---|---|
| NONE | 19.62 sec | 9.20 sec | 13.97 sec | 2364M |
| ZLIB | 15.40 sec | 14.37 sec | 33.98 sec | 624M |
| LZ4 | 11.28 sec | 6.30 sec | 6.42 sec | 1166M |
Note: peak memory usage isn't significantly different for read and write.
Possible Changes
- Using G.fileflags for two compression types is odd, should be a separate member however this is a bigger refactor of file read/writing code.
We could move these settings into a struct. - Wrap LZ4 w/ an API that hides details, similar to how zlib works.
TODO
- Thumbnail extraction utility.
It might be best to have a single implementation shared between Linux & Windows since Python doesn't support LZ4.- release/bin/blender-thumbnailer.py
- release/windows/blendthumb/src/BlenderThumb.cpp.