Page MenuHome
Differential D7673

Improve install_deps script to make it works on gentoo
AbandonedPublicDraft
Actions

Authored by Tycho Tatitsccheff (TychoTat) on May 8 2020, 7:49 PM.

Details

Reviewers
Bastien Montagne (mont29)
Summary

Context

I'm on gentoo and as the official ebuild is super outdated, i did build it using gentoo official guide.

I'm not found of using prebuild lib, both for security and performance reason, and because i got the chance to have a modern enough cpu so building from sources is not too long for me.

Here is my actual gentoo configuration:

emerge_info.txt7 KBDownload

Some useful extract from them:

gcc-9.3.0, glibc-2.30-r8, 5.4.38-gentoo

I realize that this is much more modern than what is expected on https://vfxplatform.com/ but:

  1. it would be nice to have that the install_deps script works also on new system
  2. it may help in the future

So It did invest some time and did fix the build on my side so it works.

Draft Diff

D7673: Improve install_deps script to make it works on gentoo

I'm completely open to split the diff into several pieces if you want to review each fixe one by one.

Stuff that I fixed

Boost

On gentoo I needed

diff --git a/build_files/build_environment/install_deps.sh b/build_files/build_environment/install_deps.sh
index 6a247e81148..6f58dc01b46 100755
--- a/build_files/build_environment/install_deps.sh
+++ b/build_files/build_environment/install_deps.sh
@@ -1459,7 +1459,7 @@ compile_Boost() {
     if [ ! -f $_src/b2 ]; then
       ./bootstrap.sh
     fi
-    ./b2 -j$THREADS -a $BOOST_BUILD_MODULES \
+    ./b2 --ignore-site-config -j$THREADS -a $BOOST_BUILD_MODULES \
          --prefix=$_inst --disable-icu boost.locale.icu=off install
     ./b2 --clean

That was done as per https://stackoverflow.com/questions/23013433/how-to-install-modular-boost

LLVM and LibFFI

diff --git a/build_files/build_environment/install_deps.sh b/build_files/build_environment/install_deps.sh
index 6a247e81148..6f58dc01b46 100755
--- a/build_files/build_environment/install_deps.sh
+++ b/build_files/build_environment/install_deps.sh
@@ -1972,13 +1972,13 @@ compile_LLVM() {

     cmake_d="-D CMAKE_BUILD_TYPE=Release"
     cmake_d="$cmake_d -D CMAKE_INSTALL_PREFIX=$_inst"
-    cmake_d="$cmake_d -D LLVM_ENABLE_FFI=ON"
+    cmake_d="$cmake_d -D LLVM_ENABLE_FFI=ON -D FFI_INCLUDE_DIR=/usr/lib64/libffi/include"
     cmake_d="$cmake_d -D LLVM_TARGETS_TO_BUILD=X86"
     cmake_d="$cmake_d -D LLVM_ENABLE_TERMINFO=OFF"

-    if [ -d $_FFI_INCLUDE_DIR ]; then
-      cmake_d="$cmake_d -D FFI_INCLUDE_DIR=$_FFI_INCLUDE_DIR"
-    fi
+    #if [ -d $_FFI_INCLUDE_DIR ]; then
+    #  cmake_d="$cmake_d -D FFI_INCLUDE_DIR=$_FFI_INCLUDE_DIR"
+    #fi

for some reason the _FFI_INCLUDE_DIR in place was not working. I did comment out the part modifying this and hardcoded the FFI_INCLUD_DIR path.

To find the path I used

$ pkg-config libffi --cflags-only-I
-I/usr/lib64/libffi/include

I think relying on pkg-config could help bringing cross system compatibility.

OpenVDB

diff --git a/build_files/build_environment/install_deps.sh b/build_files/build_environment/install_deps.sh
index 6a247e81148..6f58dc01b46 100755
--- a/build_files/build_environment/install_deps.sh
+++ b/build_files/build_environment/install_deps.sh
@@ -2431,8 +2431,9 @@ compile_OPENVDB() {
     fi

     if [ "$_with_built_openexr" = true ]; then
-      make_d="$make_d ILMBASE_INCL_DIR=$INST/openexr/include ILMBASE_LIB_DIR=$INST/openexr/lib"
-      make_d="$make_d EXR_INCL_DIR=$INST/openexr/include EXR_LIB_DIR=$INST/openexr/lib"
+      make_d="$make_d ILMBASE_INCL_DIR=$INST/openexr/include ILMBASE_LIB_DIR=$INST/openexr/lib64"
+      make_d="$make_d EXR_INCL_DIR=$INST/openexr/include EXR_LIB_DIR=$INST/openexr/lib64"
+      make_d="$make_d HALF_INCL_DIR=$INST/openexr/include HALF_LIB_DIR=$INST/openexr/lib64"
       INFO "ILMBASE_HOME=$INST/openexr"
     fi

for some reason openexr lib was compiled in lib64 directory

OpenCollada

OpenCollada is not compatible with modern PRCE: see https://github.com/KhronosGroup/OpenCOLLADA/issues/547

diff --git a/build_files/build_environment/install_deps.sh b/build_files/build_environment/install_deps.sh
index 6a247e81148..6f58dc01b46 100755
--- a/build_files/build_environment/install_deps.sh
+++ b/build_files/build_environment/install_deps.sh
@@ -2728,6 +2729,9 @@ compile_OpenCOLLADA() {
       git reset --hard
     fi

+    sed -i "s/typedef struct real_pcre pcre;//g" COLLADABaseUtils/include/COLLADABUPcreCompiledPattern.h
+    sed -i "s/struct real_pcre;/#include \"pcre.h\"/g" COLLADABaseUtils/include/COLLADABUPcreCompiledPattern.h
+
     # Always refresh the whole build!
     if [ -d build ]; then
       rm -rf build

I have stolen the fix from https://github.com/lasalvavida/OpenCOLLADA/pull/1

Next steps

Do you see value in fixing this ? If so I can split my fixes and work on getting it merged here.

Diff Detail

Repository
rB Blender
Branch
master
Build Status
Buildable 7964
Build 7964: arc lint + arc unit

Event Timeline

Tycho Tatitsccheff (TychoTat) created this revision.May 8 2020, 7:49 PM
Harbormaster completed remote builds in B7964: Diff 24545.May 8 2020, 7:49 PM
Tycho Tatitsccheff (TychoTat) mentioned this in T76562: Make "install_deps.sh --with-all --build-all" works on gentoo linux.May 8 2020, 7:50 PM
Brecht Van Lommel (brecht) edited the summary of this revision. (Show Details)May 9 2020, 10:47 AM
Brecht Van Lommel (brecht) added a reviewer: Bastien Montagne (mont29).
Bastien Montagne (mont29) added a comment.May 11 2020, 11:59 AM

@Tycho Tatitsccheff (TychoTat) Thanks for the patch.

I have a bit mixed feelings about it though, as-is it cannot go in obviously (would basically break most if not all other distributions). We could add some distro-specific code to fix those issues on gentoo, but the main goal of install deps is not to build the libs from source, but to use as much as possible pre-built packages.

Did you try using the make deps build target instead? It uses a different approach, building all dependencies to the official exact versions currently supported by Blender (basically generating something as close to official builds as possible)?

build_files/build_environment/install_deps.sh
1462

I would also only do that when needed (i.e., afaik, for gentoo only).

1975–1981

What is the value of that _FFI_INCLUDE_DIR for you (or is it even defined) then? Think this is a trick from deb-like distros only? So an 'else' could work much better here...

But then, cmake is supposed to handle (know) where to find those things, not sure why it would fail for the lib...

If really needed, this should be protected by distro-specific checks (and probably rather use the pkg-config command directly for gentoo then)

2434–2436

This is obviously 100% not acceptable, what is to be fixed here is those libs being installed in /lib64. This is likely something from cmake - and I would expect it to also affect all other libs built by this script?

We probably need to explicitly set INSTALL_LIB_DIR to lib in all our lib building using cmake...

2732–2734

Should be conditioned to version check of libpcre I guess?

Weird, that is a 3 years issue already... and debian testing is still on libpcre3 8.39 here!

Bastien Montagne (mont29) abandoned this revision.Mon, Jan 23, 4:06 PM

Think it's time to archive this, thanks for the patch!

Revision Contents
Changeset List

PathSize
build_files/
build_environment/
20 lines

Diff 24545

View Options

build_files/build_environment/install_deps.sh

Loading...