Deps: Ship USD Python bindings
AbandonedPublic
Actions

Authored by Ray Molenkamp (LazyDodo) on Aug 2 2022, 6:45 PM.

Details

Reviewers

Brecht Van Lommel (brecht)
Sybren A. Stüvel (sybren)

Summary

To facilitate shipping the python bindings for USD a few changes how we are dealing with libs had to be made.

Turn USD into a shared lib, so the python modules USD generates can depend on it
Turn Boost into a shared lib, since static boost::python will not work for USD
Turn TBB into a shared lib
Turn OpenSubDiv into a shared lib (Linux only, windows only supports static here)
Update a whole bunch of unrelated deps (ie OSL) that some of their deps are shared libs now.
Enable the USD python bindings
Install the shared libs with blender
Install the USD python bindings with blender

Current state:

Has some issues, but it's ready for the other platform devs to start shaking their sticks at this, given this is a big one, plan accordingly.

Windows:

Initial pass for release libs are done, build, links, installs and the single helloworld.py USD script runs ok
Todo: Debug libs
Todo: Deal with both old and new style lib folders nicely
Fix: IDiff not working due to it not finding a dll, breaking a whole bunch of tests

Linux:

Initial pass is done done on my centos VM, it builds, links, installs and the single helloworld.py USD script runs OK, had hans test my build on his workstation and ran ok there.
I helped it along the way quite a bit, i think i fixed all issues i ran into, but unsure till i try a completely clean build
Have not tried the tests yet
not super confident on the rpath shenanigans, borrowed from a diff xavier and sergey were working on for ONEAPI

Mac:

Nothing is done, but the windows/linux changes are likely a good starting point.

Diff Detail

Repository

rB Blender

Branch

tmp_usd_bindings (branched from master)

Build Status

Buildable 23239
Build 23239: arc lint + arc unit

Event Timeline

Ray Molenkamp (LazyDodo) requested review of this revision.Aug 2 2022, 6:45 PM

Ray Molenkamp (LazyDodo) created this revision.

Harbormaster completed remote builds in B23239: Diff 54312.Aug 2 2022, 6:45 PM

Ray Molenkamp (LazyDodo) edited the summary of this revision. (Show Details)Aug 2 2022, 10:38 PM

Ray Molenkamp (LazyDodo) edited the summary of this revision. (Show Details)Aug 2 2022, 10:42 PM

macOS support, tests pass.

The dynamic lib rpath and installation can use some cleanup, perhaps sharing more logic between platforms.

I'd also like to include the following changes in this update:

Make OpenVDB a dynamic lib.
Add OpenVB python module.
Remove custom usdBlender namespace?

Harbormaster completed remote builds in B23245: Diff 54345.Aug 3 2022, 2:43 PM

Make OpenVDB a dynamic lib.

Fine with me, is already dynamic on windows

Add OpenVB python module.

I took a peek at D8123 doesn't look like there is a lot we can re-use there, i'd say we abandon that one and make minor changes to this diff

Remove custom usdBlender namespace?

I'm not entirely sure what having the namespace (or not having it) brings to the party, i'll have to defer to @Sybren A. Stüvel (sybren) here

In D15604#423766, @Ray Molenkamp (LazyDodo) wrote:

I'm not entirely sure what having the namespace (or not having it) brings to the party, i'll have to defer to @Sybren A. Stüvel (sybren) here

The reason I brought it up is that this now becomes part of the public API, and it would be nice to have some consistent rules for how to handle namespaces.

If we use a custom namespace, that means a USD plug-in must be specifically built against our USD headers and will only work for Blender. In practice that may not be a big problem since USD is not part of the VFX platform, versions vary wildly between applications, and minor releases have API breaking changes. But it would be nice if a USD plug-in built against a matching USD version also works in Blender.

The reasoning for the custom namespace is this:

# This is a preventative measure that avoids possible conflicts when add-ons
# try to load another USD library into the same process space.
-DPXR_SET_INTERNAL_NAMESPACE=usdBlender

The default USD namespace includes the major and minor version, so loading different versions should have no conflicts. A potential problem is loading a USD library with the same version but different build options, but I'm not sure that's a real concern?

yeah i guess, it's one less thing that can go wrong, a minor thing, but one less thing non the less.

I pushed a tmp_libupdate_34 branch, a branch is more comfortable than 3 people working on a single diff

Updated tmp_libupdate_34 with build support for pyopenvdb.

Windows:

tested, seems to import ok

Linux:

build with one caveat :it looks for pyconfig.h in the Include folder rather than the root where it is on a raw (ie not installed) build, I manually copied it there to make it happy, but that's not really sustainable, we may have to fudge that in a post install handler for linux.
Harvesting and install in creator cmake still needs to be done

Mac: