Paths

Table of Contentst

macOS: changes to build library dependencies for arm64
ClosedPublic
Actions

Authored by Stefan Werner (swerner) on Jul 7 2020, 3:51 PM.

Details

Reviewers

Brecht Van Lommel (brecht)
Ray Molenkamp (LazyDodo)
Sergey Sharybin (sergey)

Maniphest Tasks

T78710: macOS: Support arm64

Commits

rB9715ad5acad9: macOS: Support arm64 architecture.

Summary

The changes necessary to build Blender on a Mac with arm64 CPU, using the same steps as build on a x86_64 Mac.
Except for OIDN, this will result in a feature complete but untested and most likely buggy build of Blender.

This is still work in progress - it needs a little bit more cleanup before we can commit this IMHO.

The original patches sent by Apple are archived below and are not necessary or included in this new patch:

These patches were provided by Apple.

Note this patch does not apply on latest master, it is from 5/29/2020.

There is a patch for Blender, and patches for various library dependencies attached below.

0001-png-arm64-cmake-fixe.patch687 BDownload

0001-USD-arm64.patch3 KBDownload

0001-numpy-for-arm64.patch1 KBDownload

0001-opencollada-clang-language-fixes.patch1 KBDownload

0001-openmp-arm64-fixup.patch642 BDownload

0001-blosc-arm64-fixes.patch658 BDownload

Diff Detail

Repository

rB Blender

Build Status

Buildable 8883
Build 8883: arc lint + arc unit

Event Timeline

Brecht Van Lommel (brecht) requested review of this revision.Jul 7 2020, 3:51 PM

Brecht Van Lommel (brecht) created this revision.

Harbormaster completed remote builds in B8883: Diff 26589.Jul 7 2020, 3:51 PM

Brecht Van Lommel (brecht) added projects: Platform: macOS, Platforms, Builds, Tests & Devices.Jul 7 2020, 3:51 PM

Stefan Werner (swerner) added a subscriber: Stefan Werner (swerner).Jul 7 2020, 5:01 PM

Julian Eisel (Severin) mentioned this in T78710: macOS: Support arm64.Jul 7 2020, 5:07 PM

OpenImageDenoise naturally doesn’t have an ARM port. There is however a denoiser that Apple includes in the Metal framework that we could use instead.

Julian Eisel (Severin) added a task: T78710: macOS: Support arm64.Jul 7 2020, 5:14 PM

Setting this as plan changes after code review. It doesn't mean I personally will make those changes, another developer is welcome to pick up the work here.

build_files/build_environment/CMakeLists.txt
103	This should be `if (NOT BUILD_APPLE_ARM64)`.
build_files/build_environment/cmake/clang.cmake
28–30	It's rather surprising to me that this would be arm64 specific, maybe is just needed for all architectures or not needed at all.
build_files/build_environment/cmake/harvest.cmake
35–39	These changes to the Windows build should be removed.
build_files/build_environment/cmake/openjpeg.cmake
31	This hardcoded path should be fixed.
build_files/build_environment/cmake/options.cmake
124–134	Commented out code is to be removed.
build_files/build_environment/cmake/python.cmake
65–68	Commented out code to be either used or removed.
build_files/build_environment/cmake/versions.cmake
300	Embree is disabled in this patch, so the hash should also not be changed. Presumably this was from an unfinished test.
build_files/build_environment/cmake/xr_openxr.cmake
37–39	We don't support OpenXR on Apple yet. But I assume the test needs to change to just `if (APPLE)` since `-DXR_OS_APPLE` does not seem arm64 specific.

Certainly needs some work before committing. Ideally, I'd like all scripts to obey CMAKE_OSX_ARCHITECTURES/OSX_ARCHITECTURES so that we can cross-compile.

hujin (MAXMADZZ) added a subscriber: hujin (MAXMADZZ).Jul 12 2020, 5:50 PM

macOS: Port to arm64.

All dependencies with the exception of OIDN (requires SSE4.1) now build
on arm64/macOS. With the exception of OIDN, a feature complete Blender
can build and run on the DTK.

Other developers with access to arm64 Mac hardware can now help with
testing and bug fixes. Patches are very welcome. Please respect the NDA
and do not publish benchmarks.

Harbormaster completed remote builds in B9000: Diff 26788.Jul 14 2020, 12:56 PM

Stefan Werner (swerner) commandeered this revision.Jul 14 2020, 12:56 PM

Stefan Werner (swerner) added a reviewer: Brecht Van Lommel (brecht).

In my tests, make deps on Linux and macOS/x86_64 were not broken by this change and work as before. @Ray Molenkamp (LazyDodo), can you check if this patch breaks Windows builds?

Stefan Werner (swerner) edited the summary of this revision. (Show Details)Jul 14 2020, 1:03 PM

Stefan Werner (swerner) edited the summary of this revision. (Show Details)Jul 14 2020, 1:08 PM

Do the tests pass?

build_files/build_environment/CMakeLists.txt
97	Remove indent.
build_files/build_environment/cmake/embree.cmake
22–26 ↗	(On Diff #26788)	How does this work, can you simply use SSE2 instructions somehow on macOS arm64 and it gets emulated?
build_files/build_environment/patches/python_macos.diff
1 ↗	(On Diff #26788)	Is this a backported patch from a newer Python version? If so, would be good to mention it somewhere.

Stefan Werner (swerner) marked 4 inline comments as done.Jul 14 2020, 1:45 PM

Stefan Werner (swerner) added inline comments.

build_files/build_environment/cmake/clang.cmake
42	This change may not be specific to the architecture but macOS 11.

Stefan Werner (swerner) edited the summary of this revision. (Show Details)Jul 14 2020, 1:47 PM

Stefan Werner (swerner) marked an inline comment as not done.Jul 14 2020, 2:15 PM

Stefan Werner (swerner) added inline comments.

build_files/build_environment/cmake/embree.cmake
22–26 ↗	(On Diff #26788)	This is using the Embree port for ARM from https://github.com/lighttransport/embree-aarch64/ or more precisely, a mirror that has a 3.10.0 release archive. See versions.cmake. I'm not too excited about pulling from completely separate repositories for different architectures, but I don't expect Intel to add official ARM support to Embree.
build_files/build_environment/patches/python_macos.diff
1 ↗	(On Diff #26788)	It is back ported from a PR by Apple for the latest Python. I mentioned this in the commit logs of the mac_arm64 branch, but yes, it would help a lot if I documented this in the code as well.

94% tests passed, 3 tests failed out of 52

Total Test time (real) = [redacted] sec

The following tests FAILED:
	 21 - constraints (Failed)
	 32 - cycles_denoise_animation (Failed)
	 52 - ffmpeg (Failed)

Did a clean build of the deps, no build issues on windows, but have yet to run the unit tests.

No other dependency or general build issues on the DTK from the studio. Nice to see it running!
Just noticed some linker warnings for Mantaflow - the simulation source files might need to be preprocessed differently.

Sebastián Barschkis (sebbas) retitled this revision from macOS: changes to build library dependences for arm64 to macOS: changes to build library dependencies for arm64.Jul 15 2020, 4:30 PM

Brecht Van Lommel (brecht) added inline comments.Jul 15 2020, 4:38 PM

build_files/build_environment/cmake/embree.cmake
22–26 ↗	(On Diff #26788)	Ok, we can ask Intel about their positions on this at least (CC @Sergey Sharybin (sergey)), I'm still hoping there will be ARM support in the official Embree repo at some point. For the initial commit of this patch I'd rather build without Embree then. We might end up having to maintain the changes ourselves, but then maybe it should be in the form of a patch or we host the Embree repo with modifications on git.blender.org.

Windows checks out, i had to merge master to fix of some failing tests (modifiers, constraints) but beyond that no issues.

Your failing ffmpeg test is probably an indication of one or more of the libs not being picked up, it's a pretty basic sanity check that ffmpeg has all the codecs blender counts on being there it doesn't even run them, just tests if it can instance them.

This revision is now accepted and ready to land.Jul 15 2020, 5:36 PM

Stefan Werner (swerner) added inline comments.Jul 15 2020, 8:22 PM

build_files/build_environment/cmake/embree.cmake
22–26 ↗	(On Diff #26788)	I'll leave Embree out then for now. We can still turn it on later if we need to.

Without giving absolute numbers, the difference of Embree vs BVH2 is significant. I'll have to try the ARM SIMD patch next to see what that gains us.

Closed by commit rB9715ad5acad9: macOS: Support arm64 architecture. (authored by Stefan Werner (swerner)). · Explain WhyJul 16 2020, 9:38 AM

This revision was automatically updated to reflect the committed changes.

Stefan Werner (swerner) added a commit: rB9715ad5acad9: macOS: Support arm64 architecture..

I think we should ship with Embree on Arm64 in the end. My only concern is that we host the code in a place where any Blender developer can edit it, and that we check the diff compared to Embree to ensure it can be trusted.

But maybe it's easiest to first check with Intel before we do the work.

Brecht Van Lommel (brecht) mentioned this in D9527: Use embree-aarch64 for Apple silicon.Nov 18 2020, 5:48 PM

Mehmet Oguz Derin (mehmetoguzderin) awarded a token.Dec 10 2020, 4:33 AM

Mehmet Oguz Derin (mehmetoguzderin) added a subscriber: Mehmet Oguz Derin (mehmetoguzderin).