Details

Reviewers

Brecht Van Lommel (brecht)
Campbell Barton (campbellbarton)

Maniphest Tasks

T101403: Library changes for Blender 3.4

Commits

rB2fb06407403a: Build: update various libraries for 3.4, fixing bugs and security issues
rB21deba56aaec: Build: update various libraries for 3.4, fixing bugs and security issues

Summary

This was mostly just bumping deps with known CVE's
the only exception was OIIO/OSL. OIIO needed a rebuild
anyhow because of openjpeg, so was it was no effort to
bump it to 2.4.x where some of the work @Aras Pranckevicius (aras_p) has
been doing had been landed, also I think there were some
drastic perf improvements with one of the USD sample scenes.

dep	old	new
zlib	1.2.12	1.2.13
freetype	2.11.1	2.12.1
openimageio	2.3.13.0	2.3.20.0
OSL	1.11.17.0	1.12.6.2
python	3.10.2	3.10.8
openjpeg	2.4.0	2.5.0
ffmpeg	5.0	5.1.2
sndfile	1.0.28	1.1.0
xml2	2.9.10	2.10.3
expat	2.4.4	2.4.9
openssl	1.1.1g/i	1.1.1q
sqlite	3.31.1	3.37.2
openpgl	0.3.1-beta	0.4.0-beta

This diff is mostly just so we can get an easy overview of the
changes being done, please do any work in the tmp_libs_34 branch.
(and update this diff occasionally)

Notable changes:

AOM : the hack we had in place to make it not detect
pthreads on windows no longer worked with a more
recent cmake version. Disabled pthreads with a diff
(windows only)

Python:
Python had an embedded copy of zlib 2.1.12 swapped it
out for our 2.1.13 copy with some folder manipulation
on windows.

Freetype:
Freetype was harbouring a copy of zlib 2.1.12 as well,
so that had to end.

FFMpeg:
The patch we had in place for D11796 no longer applies,
and it doesn't look like it is needed any-more, but i
was unable to verify this since the original problem
only showed on mac.

There is a new patch to deal with simple_idct.asm
generating an object file with no sections in it,
upsetting strip, causing a build error. This is a
backport from an upstream commit.

SQLITE:
They changed their filenames a bit, given python takes
care of these on windows, i hope i picked the right one.

OSL:
OSL has an extra lib now: oslnoise. this may need
adjustments in the platform/findXXXX cmake files for that.

TINYXML:
was still being downloaded, but nothing seemingly uses
it, as there are no build scripts for it. Removed it
from versions/download.cmake

Diff Detail

Repository

rB Blender

Branch

tmp_libs_34

Build Status

Buildable 24289
Build 24289: arc lint + arc unit

Event Timeline

Ray Molenkamp (LazyDodo) requested review of this revision.Oct 17 2022, 7:52 PM

Ray Molenkamp (LazyDodo) created this revision.

Harbormaster completed remote builds in B24289: Diff 56874.Oct 17 2022, 7:52 PM

Ray Molenkamp (LazyDodo) added a task: T101403: Library changes for Blender 3.4.Oct 17 2022, 7:53 PM

Building the branch is asking for patches for external_sqlite & external_sndfile (ba4cc1693111e633fcf3fa9588e8f2cbad66ffb0).

Is this intended? (it prompts for the missing patch and can be skipped).

Running into a build error with external_wayland, P3261

Running into a build error with external_tiff, P3262

i just spun up my centos VM, i see the wayland one as well, we bumped the xml lib, feels like a -lm is missing somewhere, tiff builds fine here, try a rm -rf build/tiff` and see if it goes away

Fix building wayland with XML2

Harbormaster completed remote builds in B24296: Diff 56891.Oct 18 2022, 9:10 AM

@Brecht Van Lommel (brecht) both patches that fail to apply are for macOS:

I couldn't tell if the external_sndfile patch is still needed (checking changes to sndfile it seems possible they're not).

@Ray Molenkamp (LazyDodo)

The patch for sqlite would apply (updated patch P3263), but this patch uses sqlite-amalgamation-${SQLLITE_LONG_VERSION}.zip which only includes the source files (not the build system files). Won't Linux/macOS need the configure command to build this library?

In D16269#443292, @Campbell Barton (campbellbarton) wrote:

@Ray Molenkamp (LazyDodo)
The patch for sqlite would apply (updated patch P3263), but this patch uses sqlite-amalgamation-${SQLLITE_LONG_VERSION}.zip which only includes the source files (not the build system files). Won't Linux/macOS need the configure command to build this library?

from the patch description:

SQLITE:
They changed their filenames a bit, given python takes care of these on windows, i hope i picked the right one.

sounds like i chose.... poorly.. can you try sqlite-autoconf-${SQLLITE_LONG_VERSION}.zip ?

I fixed the various issues mentioned here in the branch, builds fine on macOS now.

Remove unnecessary sndfile/sqlite patches
Add patch to fix ssl and osl compile errors on macOS
Download full sqlite source code with autoconf
Add oslnoise library, unify macOS find package code
Wayland math linking fix from Campbell

Update from branch.

Harbormaster completed remote builds in B24306: Diff 56905.Oct 18 2022, 4:41 PM

Ray Molenkamp (LazyDodo) edited the summary of this revision. (Show Details)Oct 18 2022, 4:43 PM

Ray Molenkamp (LazyDodo) edited the summary of this revision. (Show Details)Oct 18 2022, 5:02 PM

Ray Molenkamp (LazyDodo) mentioned this in T101403: Library changes for Blender 3.4.Oct 18 2022, 5:37 PM

@Ray Molenkamp (LazyDodo) and I made some further changes in the branch so make cve_check ignores CVEs not affecting Blender. I also patched gmp with a security fix not yet in a release.

The end result is there are no more CVEs to be resolved.

This revision is now accepted and ready to land.Oct 18 2022, 7:42 PM

Hold off landing i'd like to make sure missing oslnoise is not gonna make trouble for the current libs that lack it.

Add newline to end of diff (causes error applying the patch)

This revision is now accepted and ready to land.Oct 19 2022, 3:52 AM

Harbormaster completed remote builds in B24326: Diff 56936.Oct 19 2022, 3:52 AM

deps_builder: add missing zlib dep to python
cmake/Windows: 3.3/3.4 lib folder compatibility

Harbormaster completed remote builds in B24342: Diff 56965.Oct 19 2022, 7:48 PM

I can't accept since it's my own patch, but I'm good to go on windows for this, can land without landing the windows libs first, as it'll build with both 3.3 and 3.4 lib folders.

Brecht Van Lommel (brecht) added inline comments.Oct 19 2022, 7:54 PM