Page MenuHome

Deps builder: Initial pass as deps update for 3.4
ClosedPublic

Authored by Ray Molenkamp (LazyDodo) on Oct 17 2022, 7:52 PM.

Details

Summary

This was mostly just bumping deps with known CVE's
the only exception was OIIO/OSL. OIIO needed a rebuild
anyhow because of openjpeg, so was it was no effort to
bump it to 2.4.x where some of the work @Aras Pranckevicius (aras_p) has
been doing had been landed
, also I think there were some
drastic perf improvements with one of the USD sample scenes.

depoldnew
zlib1.2.121.2.13
freetype2.11.12.12.1
openimageio2.3.13.02.3.20.0
OSL1.11.17.01.12.6.2
python3.10.23.10.8
openjpeg2.4.02.5.0
ffmpeg5.05.1.2
sndfile1.0.281.1.0
xml22.9.102.10.3
expat2.4.42.4.9
openssl1.1.1g/i1.1.1q
sqlite3.31.13.37.2
openpgl0.3.1-beta0.4.0-beta

This diff is mostly just so we can get an easy overview of the
changes being done, please do any work in the tmp_libs_34 branch.
(and update this diff occasionally)

Notable changes:

AOM : the hack we had in place to make it not detect
pthreads on windows no longer worked with a more
recent cmake version. Disabled pthreads with a diff
(windows only)

Python:
Python had an embedded copy of zlib 2.1.12 swapped it
out for our 2.1.13 copy with some folder manipulation
on windows.

Freetype:
Freetype was harbouring a copy of zlib 2.1.12 as well,
so that had to end.

FFMpeg:
The patch we had in place for D11796 no longer applies,
and it doesn't look like it is needed any-more, but i
was unable to verify this since the original problem
only showed on mac.

There is a new patch to deal with simple_idct.asm
generating an object file with no sections in it,
upsetting strip, causing a build error. This is a
backport from an upstream commit.

SQLITE:
They changed their filenames a bit, given python takes
care of these on windows, i hope i picked the right one.

OSL:
OSL has an extra lib now: oslnoise. this may need
adjustments in the platform/findXXXX cmake files for that.

TINYXML:
was still being downloaded, but nothing seemingly uses
it, as there are no build scripts for it. Removed it
from versions/download.cmake

Diff Detail

Repository
rB Blender
Branch
tmp_libs_34
Build Status
Buildable 24342
Build 24342: arc lint + arc unit

Event Timeline

Ray Molenkamp (LazyDodo) requested review of this revision.Oct 17 2022, 7:52 PM
Ray Molenkamp (LazyDodo) created this revision.

Building the branch is asking for patches for external_sqlite & external_sndfile (ba4cc1693111e633fcf3fa9588e8f2cbad66ffb0).

Is this intended? (it prompts for the missing patch and can be skipped).


  • Running into a build error with external_wayland, P3261
  • Running into a build error with external_tiff, P3262

i just spun up my centos VM, i see the wayland one as well, we bumped the xml lib, feels like a -lm is missing somewhere, tiff builds fine here, try a rm -rf build/tiff` and see if it goes away

  • Fix building wayland with XML2

@Brecht Van Lommel (brecht) both patches that fail to apply are for macOS:

I couldn't tell if the external_sndfile patch is still needed (checking changes to sndfile it seems possible they're not).

@Ray Molenkamp (LazyDodo)

The patch for sqlite would apply (updated patch P3263), but this patch uses sqlite-amalgamation-${SQLLITE_LONG_VERSION}.zip which only includes the source files (not the build system files). Won't Linux/macOS need the configure command to build this library?

@Ray Molenkamp (LazyDodo)
The patch for sqlite would apply (updated patch P3263), but this patch uses sqlite-amalgamation-${SQLLITE_LONG_VERSION}.zip which only includes the source files (not the build system files). Won't Linux/macOS need the configure command to build this library?

from the patch description:

SQLITE:
They changed their filenames a bit, given python takes care of these on windows, i hope i picked the right one.

sounds like i chose.... poorly.. can you try sqlite-autoconf-${SQLLITE_LONG_VERSION}.zip ?

I fixed the various issues mentioned here in the branch, builds fine on macOS now.

  • Remove unnecessary sndfile/sqlite patches
  • Add patch to fix ssl and osl compile errors on macOS
  • Download full sqlite source code with autoconf
  • Add oslnoise library, unify macOS find package code
  • Wayland math linking fix from Campbell

@Ray Molenkamp (LazyDodo) and I made some further changes in the branch so make cve_check ignores CVEs not affecting Blender. I also patched gmp with a security fix not yet in a release.

The end result is there are no more CVEs to be resolved.

This revision is now accepted and ready to land.Oct 18 2022, 7:42 PM
Ray Molenkamp (LazyDodo) planned changes to this revision.Oct 18 2022, 8:12 PM

Hold off landing i'd like to make sure missing oslnoise is not gonna make trouble for the current libs that lack it.

  • Add newline to end of diff (causes error applying the patch)
This revision is now accepted and ready to land.Oct 19 2022, 3:52 AM
  • deps_builder: add missing zlib dep to python
  • cmake/Windows: 3.3/3.4 lib folder compatibility

I can't accept since it's my own patch, but I'm good to go on windows for this, can land without landing the windows libs first, as it'll build with both 3.3 and 3.4 lib folders.

build_files/cmake/platform/platform_win32.cmake
686

Should this be changed to 2.4?

  • cmake: fix openjpeg version on windows.
build_files/cmake/platform/platform_win32.cmake
686

yes

Ok, seems this is fine to commit then.

Haven't heard from campbell yet, so i'll hold off landing for now just in case he still has open issues

Campbell accepted the revision so I think we're good.

I'll commit this as a few separate commits so it's easier to backport the relevant parts to 3.3 LTS.