Page MenuHome

PyAPI: temporary context override support
ClosedPublic

Authored by Campbell Barton (campbellbarton) on Nov 5 2021, 12:47 PM.
Subscribers
None
Tokens
"Burninate" token, awarded by zanqdo."Love" token, awarded by bobalazek."Love" token, awarded by kursadk."Like" token, awarded by Severin.

Details

Summary

Support a way to temporarily override the context from Python.

  • Added method Context.temp_override context manager.
  • Special support for windowing variables "window", "area" and "region", other context members such as "active_object".
  • Nesting context overrides is supported.
  • Previous windowing members are restored when the context exists unless they have been removed.
  • Overriding context members by passing a dictionary into operators in bpy.ops has been deprecated and warns when used.

This allows the window in a newly loaded file to be used, see: T92464


Motivation for this Patch

The initial motivation was to let script authors select a window after loading a file to continue operations after a file is loaded and the context's window is cleared.

However this has wider implications we should consider, one is that script authors will be able to run operators in contexts that currently don't use an active window - Python timers for example.

This can be seen as a benefit but also allows for operators being executed outside the main event loop - which we might want to avoid.

If there are contexts where this shouldn't be possible we could explicitly prevent it too.

This patch is a test for overriding some of the context members, it could be extended on as needed.


Notes:

  • bpy.context.temp_override name could change.
  • This could be extended to override other kinds of data, although screen/scene are tried to the window internally.
  • pyrna_struct_as_ptr and related API's should be committed separately.

This example script adds a cube in the non-active window (which may have a different scene).

import bpy
from bpy import context
wm = context.window_manager
win_active = context.window
win_other = None
for win_iter in wm.windows:
    if win_iter != win_active:
        win_other = win_iter
        break

# Add cube in the other window.
with context.temp_override(window=win_other):
    bpy.ops.mesh.primitive_cube_add()

Diff Detail

Repository
rB Blender
Branch
TEMP-CONTEXT-OVERRIDE (branched from master)
Build Status
Buildable 18498
Build 18498: arc lint + arc unit

Event Timeline

Campbell Barton (campbellbarton) requested review of this revision.Nov 5 2021, 12:47 PM
Campbell Barton (campbellbarton) created this revision.
  • Correct _PyArg_ParseTupleAndKeywordsFast string formatting
  • Clear the window in the case of file load.

Interesting idea. Still have to look at it in more detail, but makes sense generally. I wonder if this should be the preferred method for overriding the context if it is accepted (instead of creating a`dict` that contains the overrides).
As for the naming, I think just override is better than temp_override. The fact that it returns a context manager indicates that it is supposed to be used in a temporary scope.

Found it funny how the patch uses a context manager to actually manage context.

  • Rename temp_override to override

While not opposed to the general idea here, I wonder why current context override is not working/good enough in that case?

And am not very happy with using just override as name here either, we already use override in a very different meaning in ID data... Would suggest to either be more verbosely explicit (context.context_override() ?), or use a different name altogether for context override (supersede ? replace ?).

We already suffer from our multi-usages of collection e.g., think we should try to avoid that kind of issues as much as possible?

While not opposed to the general idea here, I wonder why current context override is not working/good enough in that case?

Two reason:

  • Those overrides are only used when calling operators, it's possible we may want to set the context outside of operators, although we could support this exact style of overriding context members with a context manager (without actually setting the active window, area... etc).
  • These set up overrides that work by accessing from the context will override. e.g. CTX_data_active_object(C) where as using BASACT(view_layer) wont.

    In principle it could be argued that's the context should always be used removing the need for this feature.

    In practice there is enough code that uses direct access to window/screen/scene, e.g. WM_window_get_active_scene(C) / WM_window_get_active_screen(C) .. etc.

    Temporarily swapping out windowing context is done quite a lot in Blender's C code: P2725.

We could change how operator overrides work (with this API https://docs.blender.org/api/master/bpy.ops.html#overriding-context so these changes actually temporarily modify the values of C.wm.* for example. This has pros and cons and could back-fire in some cases (specifically when restoring the original value overwrites a change which the user wished to keep from the resulting operation) - although we could detect this and only restore values that are unmodified by the operator - it feels quite fragile.

Another down side is that only some members will be temporarily assigned (at least initially), adding support for members has the potential for subtle changes in behavior that could break scripts.

Note that if this were done we would still want a context manager to override the context members outside the operator (not a problem, just mentioning).

And am not very happy with using just override as name here either, we already use override in a very different meaning in ID data... Would suggest to either be more verbosely explicit (context.context_override() ?), or use a different name altogether for context override (supersede ? replace ?).

We already suffer from our multi-usages of collection e.g., think we should try to avoid that kind of issues as much as possible?

Agree although logically it _is_ an override, it's not all that helpful to use that term in this context (no pun inten... sigh :) ).

Possible alternative could be "state", e.g. with context.temp_state(..): ....

Open topics:

  • In principle, would it be better to follow the current context override API? (assuming the details can be figured out).
  • Would this be limited to window manager members or would we eventually want to extend this to active object, bones .. etc?
  • Might we be better off with a more limited solution that handles the case of loading a file not having an active window (personally rather more general solution, mentioning for completeness).

Perhaps best to discuss this in chat, since there are various ways we could handle this.

In general, getting something like this to work would be quite nice. I've seen plenty of cases where a general way to override context would've made things easier.

My main worry is also code that mixes context access with direct access. I just see too many things that can break, and it's a bit fragile.

In principle it could be argued that's the context should always be used removing the need for this feature.

I'd strongly argue against this. IMO context should be passed around as little as possible, because otherwise there is going to be a lot of code depending on it, it gets annoying and dirty. We'll have to pass it around through multiple layers just because some function deep down the call stack requires it, and we'll end up with more evil_C usage and access. It's also not clear what context functions expect -- "is the context region required, or just the area?? Oh, this actually uses the scene, I didn't expect that..."
What I'd like to move towards is, operators use context to extract the data to operate on, then they hand that as input to functions. Make it clear what functions operate on.


Of course another thing we can do, is just add more functions to execute instead of the operators.

@Julian Eisel (Severin) Not sure I get your point about not using context as much as possible... I would rather say the opposite, context should be used as much as possible (just like

Had a chat with Campbell this morning, we considered quite a few alternative solutions, conclusions are:

  • naming: temp_override is fine, this is a very specific use-case so it's fairly unlikely to create confusion with the 'data' override cases (library override, dynamic override, etc.).
  • Proposed solution is fine, but should be extended to cover also the 'random keyword' context override currently provided by the override_context positional argument of operator calls, Then this later option can be deprecated and removed in a few releases.
  • If current behavior to always reset to original values on exit is kept, it should be clearly documented, as it can have unexpected effects on results of operators that would e.g. change the active window/area/region (since it would undo that change).
  • Might need some closer check on modal operators, although it should work in a similar way as current behavior when calling with an override_context defined (i.e. the operator is responsible to store whichever data it needs to be persistent over its modal execution).
  • SPDX headers, include RNA_prototypes.h

Update based on feedback from @Bastien Montagne (mont29).

  • Add documentation & examples.
  • Add support for overriding arbitrary context members.
  • Add support for nesting overrides.
  • Deprecate passing overrides to methods in bpy.ops.

Documentation generation needed to be updated to include Context.temp_override, rB775f0d76d55b2977c30dcd0f9306437ae520d63f.

  • Minor wording tweak for the methods doc-string.

Functionality wise LGTM now, and could not spot any issue with the code itself.

doc/python_api/examples/bpy.types.Context.temp_override.1.py
14 ↗(On Diff #50558)

The only
?

17 ↗(On Diff #50558)

such as

doc/python_api/examples/bpy.types.Context.temp_override.2.py
3 ↗(On Diff #50558)

F

source/blender/python/intern/bpy_rna_context.c
53

supports

This revision is now accepted and ready to land.Apr 19 2022, 3:29 PM
  • Correct accidental of uncommitted changes in the master branch.
  • Correct double back-tick quoting in doc-strings

As for the naming, I think just override is better than temp_override. The fact that it returns a context manager indicates that it is supposed to be used in a temporary scope.

Not ignoring your comment but mont29 felt strongly about this, and personally I'm not that fussed.

Update as pyrna_struct_as_ptr was committed separately