I added basic support on the buildbot for building the bpy module, available as an option for patch and branch builds.

Example run:
https://builder-uatest.blender.org/admin/#/builders/169/builds/12
https://builder-uatest.blender.org/download/patch/

Linux and Windows are failing, tests are failing, the package name is wrong, etc. But maybe all of that is fixable with just changes to the Blender repository.

This is an update to the script that extracts the Python & Blender version P3190.

Some of the utility functions could be moved into make_utils.py.

https://builder-uatest.blender.org/admin/#/builders/170/builds/26

Suggesting some further changes, although additional meta-data can be handled once in master too.

• *picky* (stick to one style of quote), better do here to avoid noise in master (maybe best use " to stick to Blender's Python code style).
• Failure could use sys.exit(1) instead of early returns, so automation can detect failure.
• Additional meta-data (not so urgent but should probably include at minimum "License": license='GPL-3.0'), see: https://github.com/TylerGubala/blenderpy/blob/master/setup.py for other possible inclusions.

Builds passing now, except some unit tests on Windows. Close enough to merge this I think.
https://builder-uatest.blender.org/admin/#/builders/169/builds/23

I added @Ray Molenkamp (LazyDodo) as reviewer for the Windows build configuration changes,

changes look sane to me, couldn't get the wheel to build but that's likely something i screwed up, did not spend a whole lot of time on it

k:\BlenderGit\blender>K:\BlenderGit\lib\win64_vc15\python\310\bin\python.exe build_files\utils\make_bpy_wheel.py --build-dir k:\BlenderGit\bpy_2022\ --output-dir k:\blendergit\wheel k:\BlenderGit\bpy_2022\bin
K:\BlenderGit\lib\win64_vc15\python\310\lib\site-packages\setuptools\dist.py:487: UserWarning: Normalizing '3.4.0-alpha' to '3.4.0a0'
  warnings.warn(tmpl.format(**locals()))
usage: make_bpy_wheel.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: make_bpy_wheel.py --help [cmd1 cmd2 ...]
   or: make_bpy_wheel.py --help-commands
   or: make_bpy_wheel.py cmd --help

error: invalid command 'bdist_wheel'

In D15957#434589, @Ray Molenkamp (LazyDodo) wrote:

error: invalid command 'bdist_wheel'

This seems to be because the wheel package is not installed (pip install wheel). Probably the Python used to execute the script on the buildbot has it, but not the Python in the lib folder. We could bundle it for convenience.

In D15957#434595, @Ray Molenkamp (LazyDodo) wrote:

looking at the test failures on the bots, it's likely missing some dll folder in its env and the binary failing to load

Yeah, I might just disable all the gtests for now. At least on the buildbot this is effectively covered by the regular Blender build.

.... At the point you're doing all of this work, are you sure you don't want to use pip + setuptools + scikit-build?

• It'll automatically deal with reading from/re-using cmake cache, and handling the linking options required for python.
• It'll automatically name the wheel file, and handles platform-specific strangeness (the manylinux wheel likely needs both manylinux2014 and manylinux_2_17 for instance, if we're building on CentOS 7).
• It's the standard way to build a python wheel, especially a native python wheel, so it will be more familiar to future maintainers who deal with python.
• There's options to automatically discover the version from git, if that's desired.

The only things needed to be installed before calling such a script is python (specifically the version of python to build/distribute against; that is, the python in the lib directory), and pip. Pip would handle auto-downloading the necessary additional build-time tools (it creates a single-use venv in /tmp by default, but there's ways to control that).

yeah i'd be fine with most of the tests being disabled for this build, the import_bpy.py test is the most important one here, as long as that one passes i think we're in good shape

In D15957#434606, @Stephen Imhoff (Clockwork-Muse) wrote:

.... At the point you're doing all of this work, are you sure you don't want to use pip + setuptools + scikit-build?

Basically yes, in the end I think this is still simpler than trying to bend our build pipeline to scikit-build.

The code that would have been replaced by scikit-build ended up working basically immediately, it's the integration with everything else that took most time to debug, and extra abstraction layers make that harder.

• It'll automatically name the wheel file, and handles platform-specific strangeness (the manylinux wheel likely needs both manylinux2014 and manylinux_2_17 for instance, if we're building on CentOS 7).

My understanding is that manylinux_2_17 is the future proof solution. manylinux2014 may be needed for older Python and pip versions, but we're not making wheels for those.