Page MenuHome

fix T54152 --env-system-scripts doesn't work on windows.
ClosedPublic

Authored by Ray Molenkamp (LazyDodo) on Feb 24 2018, 5:30 PM.

Details

Summary

Couple of issues at play here,

  1. windows weirdness: getenv and SetEnvironmentVariable don't play together , things set by SetEnvironmentVariable can only be retrieved by GetEnvironmentVariable , getenv will always return the variables as they were at process initialization. [1]
  1. BKE_appdir_folder_id_ex checks the standard location of the scripts folder (ie 2.79/scripts) first before considering the environment override .

[1] http://techunravel.blogspot.ca/2011/08/win32-env-variable-pitfall-of.html

Diff Detail

Repository
rB Blender
Branch
arcpatch-D3079_1
Build Status
Buildable 2012
Build 2012: arc lint + arc unit

Event Timeline

Generally LGTM, any reason not to poison getenv to make sure it's not used by accident?

source/blender/blenlib/BLI_path_util.h
42–44

Should probably be in BLI_os_util.h (these aren't paths), but fine for now.

source/blender/blenlib/intern/path_util.c
1203–1206

Comment seems wrong?

Ray Molenkamp (LazyDodo) marked an inline comment as done.Feb 25 2018, 5:49 PM

Generally LGTM, any reason not to poison getenv to make sure it's not used by accident?

multiple reasons,

  1. never heard of it, grepped the code base, seems like it's rarely used, and gcc specific.
  1. I don't see a nice way to do it. poison is gcc specific, msvc has #pragma deprecated which kinda does the same thing but emits a warn, but i guess we could add /we4995 to the list of warnings promoted to errors in the main CMakelists.txt. so in theory we could make a macro like this
#ifdef WIN32
#    define POISON(x) __pragma(deprecated(x))
#else
#    define POISON(x) __pragma(GCC poison x)
#endif

and dump it in a header of our liking. and use POISON(getenv) however now there's a new set of problems.

a) in BLI_getenv we still very much like to call it for non windows platforms.

b) Any other deprecated functions such as GetVersionExA (called from system header multimon.h so we can't go in and fix it) will now error instead of warn..

so neat idea, but it seems a tiny bit like opening a can of worms.

Avoiding getenv can be handled in a separate patch,
note that its possible to poison a function and ignore it in a spesific location if we want, or we could just not poison it for the file which calls it.


Reason to suggest this is it's easy for issues to creep back in if a developer forgets we have our own version of this function, this might take years to notice.. users/developers might loose some hours to research and find its exactly the same bug we already solved.

Looked further into a possible deprecation framework. And there's the following issues.

  1. No unified way to do this across compilers.
  2. There is some way to do this (ie attribute((deprecated)) or #pragma poison for gcc and __declspec(deprecated) for msvc) this is really only applicable to code we control. deprecating functions in the crt is troublesome.
  3. even if 2 would work, there's no real way to enforce it, if someone doesn't include the header for deprecation he'll still happily be able to call getenv. There is a way to force include a header for most compilers, but that in turn might break any third party code that we build.
  4. no exceptions, it's nearly impossible to make an exception,and we still very much would like to call getenv inside BLI_getenv for some of the platforms.

I feel this is more a task for something like a lint tool (maybe clang-tidy?) than something we can properly solve with some header hackery.

I don't see a way to turn this into a nicely supported framework that's not a giant kludge of hacks and ductape, either way it's well beyond the scope of fixing T54152.

I don't think its necessary to have a deprecation framework or an all-or-nothing approach. Solving this for GCC/Clang is enough that these problems don't get into a release.
MSVC support would be nice, but IIRC it doesn't support features needed for this.

It could be as simple as:

  • Define functions we don't allow as error defines.
  • Explicit use can undef them.

Or we could use a regex search which runs periodically (ignoring comments and accounting for a handful of exceptions).

Ray Molenkamp (LazyDodo) updated this revision to Diff 10302.EditedMar 30 2018, 6:36 PM
  • bli_path_util: deprecate the use of getenv.

added a mechanism to deprecate I think the DO_PRAGMA macro is right for gcc,
but can't test it, also unsure about the __MESSAGE macro it's using the ms convention of

sourcefile(line) : Error : message

but I'm unsure if gcc's formatting is identical.

so if a gcc user could give it a shake that be appriciated.

Ray Molenkamp (LazyDodo) planned changes to this revision.May 26 2018, 7:21 PM

doesn't work on linux

Removed deprecation code.

Please review as is, the deprecation thing has wasted enough time, can't find a nice way that'll work with all compiler configurations, and this bug is holding back other windows things.

Brecht Van Lommel (brecht) requested changes to this revision.Aug 8 2018, 8:04 PM

I think the BLI_getenv changes are fine, with or without poisoning. We already have BLI_setenv so it's not like we're making things much worse.

Swapping the order of get_path_system() and get_path_local() seems risky to me. I can't think of specific cases that will break, but this code is fragile and there's many ways to install on different platforms.

I think we need to check BLENDER_SYSTEM_DATAFILES before local. To make BLENDER_USER_DATAFILES have an effect with portable installs it should be done a little earlier. So I suggest to move the environment variable path testing to a new function call get_path_env() and order the tests like this:

get_path_env(BLENDER_USER_DATAFILES) 
get_path_user()
get_path_env(BLENDER_SYSTEM_DATAFILES) 
get_path_local()
get_path_system()
This revision now requires changes to proceed.Aug 8 2018, 8:04 PM
  • Merge remote-tracking branch 'origin/master' into arcpatch-D3079_1
  • update with feedback.

ah crap, i merged master and a bunch of comment changes leaked in, no idea wtf.... i'm blaming arc here.

Ray Molenkamp (LazyDodo) planned changes to this revision.Sep 3 2018, 7:50 PM
  • fix unintended changes to physics_fluid.c
This revision is now accepted and ready to land.Sep 3 2018, 8:14 PM

Committed rBc13b2a2504393371e32a4fb4c01cfad8d7cc929e

get_path_environment had strange comment wrapping, corrected.