Page MenuHome

macOS Deps: NumPy: Remove buggy Accelerate framework
ClosedPublic

Authored by Ankit Meel (ankitm) on Feb 9 2021, 12:42 PM.

Details

Summary

Deprecated since version 1.20: The native libraries on
macOS, provided by Accelerate, are not fit for use in NumPy
since they have bugs that cause wrong output under
easily reproducible conditions.

[1]

Building NumPy from source with default options of deps_builder
causes it to link against Accelerate framework which is buggy and
raises a warning mentioned in [2].

So either we build OpenBLAS in deps also and set appropriate
env variables [3] suggested in [1]. Or download NumPy from pip
and never allow it to build it from source while installing.

A nice side effect of making this change is that find_package(Python3...)
can be used instead of hardcoding version/paths etc in platform_apple.cmake

[1] https://numpy.org/doc/stable/user/building.html#disabling-atlas-and-other-accelerated-libraries
[2] https://github.com/numpy/numpy/issues/15947
[3] https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/numpy.rb

Diff Detail

Repository
rB Blender
Branch
numpy_pip (branched from master)
Build Status
Buildable 12861
Build 12861: arc lint + arc unit

Event Timeline

Ankit Meel (ankitm) requested review of this revision.Feb 9 2021, 12:42 PM
Ankit Meel (ankitm) created this revision.
Ankit Meel (ankitm) edited the summary of this revision. (Show Details)

From what I can tell, there are no current numpy binaries on pip for macOS arm64:
https://pypi.org/project/numpy/#files

So we might need to build with OpenBLAS ourselves?

Ankit Meel (ankitm) added a comment.EditedFeb 9 2021, 8:43 PM

Didn't even think of that..
From a quick search: https://github.com/numpy/numpy/issues/18143, here's a convenience it's NumPy 1.20, not 1.19: https://anaconda.org/conda-forge/numpy/files
Will try building OpenBLAS and NumPy soon.

  • New attempt with OpenBLAS
  • Fix link issues with Apple GCC
  • Use rpath for libopenblas in shared libraries of NumPy.
Ankit Meel (ankitm) added a comment.EditedFeb 11 2021, 7:37 PM

This is WIP, but I'd be happy if wrong direction is pointed out early.
Where I'm taking this is:

  • The libopenblas dylib is modified to have an id @rpath/libopenblas.dylib and aliases will sort themselves out.
  • NumPy's shared libraries like some in [1] dynamically load libopenblas. So an rpath entry is added to all of them: @loader_path/../.dylibs.
  • All openblas libraries are moved to the above rpath (in NumPy folder) in the after_install step of NumPy.

What remains:

  • several options (like 32 bit vs 64 bit BINARY, oldest TARGET architecture, boolean INTERFACE64 for some symbol renaming) in OpenBLAS.
  • Adding an rpath to libopenblas.dylib where all libraries in [2] can be found.
  • Moving the libraries mentioned in the last point to a nice place. Is there a way around shipping libgfortran, libgcc_s and libquadmath?
  • Adding a test to see if NumPy works properly. import numpy; numpy.show_config() seems enough to see whether it will work. Do we run the full test suite?

[1] _multiarray_tests.cpython-39-darwin.so , _multiarray_umath.cpython-39-darwin.so, _umath_tests.cpython-39-darwin.so`, _umath_linalg.cpython-39-darwin.so, lapack_lite.cpython-39-darwin.so
[2]

>>> otool -L Release/python/lib/python3.9/site-packages/numpy/.dylibs/libopenblas.0.dylib
        @rpath/libopenblas.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/local/opt/gcc/lib/gcc/10/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
	/usr/local/opt/gcc/lib/gcc/10/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/local/lib/gcc/10/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

Wow, this seems very complicated, I had no idea OpenBLAS would be so hard to build.

We could wait a while to see if arm64 binaries get added to pip, and only commit the x86-64 one using the original patch. It seems possible that there will be arm64 binaries in time before the 2.93 release.
https://github.com/numpy/numpy/issues/18143

If you can get it working and don't mind spending the time that's fine, but don't feel like we have to do this.

I just built the deps for macOS arm64 and numpy in Blender works just fine. So it seems the bug with the Accelerate framework is not present on macOS arm64.

So in theory, we could just download numpy from pip for x86. And leave arm64 as is.

@Sebastián Barschkis (sebbas) In the new svn libraries, openblas is being referred to it seems:

otool -L *.so
_umath_linalg.cpython-39-darwin.so:
	@loader_path/../.dylibs/libopenblas.0.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
lapack_lite.cpython-39-darwin.so:
	@loader_path/../.dylibs/libopenblas.0.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

But there's no .dylib folder here.. did SVN ignore that too ? The default order is "MKL > BLIS > OpenBLAS > ATLAS > BLAS (NetLIB)". So if OpenBLAS was available on your system, maybe it was just used instead of Accelerate. I don't think the bug is just gone when the people on numpy-github say that their bug reports to Apple were not attended to, and NumPy 1.20 deprecated Accelerate too.
Could you check the output of numpy.show_config() ?

We could wait a while to see if arm64 binaries get added to pip, and only commit the x86-64 one using the original patch. It seems possible that there will be arm64 binaries in time before the 2.93 release.

That will be awesome.

If you can get it working and don't mind spending the time that's fine, but don't feel like we have to do this.

I can't guarantee that even after changing all these id-s and rpath-s, NumPy will be portable. There's a high chance that I'll miss a library which will be absent on other people's OS and thus will cause "image not loaded"/ "symbol not found" errors.
I'd like to stop working on the hard way (:

Here's the current diff for easy access:


I'll revert it to the old way (pip install ...).

@Ankit Meel (ankitm) From macOS arm build:

>>> numpy.show_config()
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
atlas_3_10_blas_threads_info:
  NOT AVAILABLE
atlas_3_10_blas_info:
  NOT AVAILABLE
atlas_blas_threads_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
accelerate_info:
    extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
blas_opt_info:
    extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
  NOT AVAILABLE
openblas_clapack_info:
  NOT AVAILABLE
flame_info:
  NOT AVAILABLE
atlas_3_10_threads_info:
  NOT AVAILABLE
atlas_3_10_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE
atlas_info:
  NOT AVAILABLE
lapack_opt_info:
    extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]

Bring back the bygones.

Forgot to mention in the last post: gfortran is from GNU GCC.. one more dependency (:

build_files/build_environment/cmake/numpy.cmake
34–57

if((APPLE AND ("${CMAKE_OSX_ARCHITECTURES}" STREQUAL "x86_64"))

  • Check for x86_64

i'd really like this to live in python_site_packages.cmake so all pip related things are in a single location. if we're grabbing binaries from pip best to do it for all platforms

  • Fix copy paste typo
  • Move to python_site_packages. Add option USE_PIP_NUMPY. Add --no-cache-dir

Someone had to start the process of using pip.

  • Typo "=" -> "=="
This revision is now accepted and ready to land.Feb 15 2021, 7:17 PM