Page MenuHome

Initial work for std::filesystem transition
AbandonedPublic

Authored by Julian Eisel (Severin) on Aug 3 2021, 6:32 PM.

Details

Summary

This patch is more meant as a proof-of-concept for T90379, although OTOH if there are no bigger objections, this could already be reviewed properly. The changes to the existing BLI code are more for initial testing. They should be committed separately and covered well in unit tests to avoid regressions.

  • Add ghc::filesystem to extern/ as platform compatible replacement for std::filesystem.
  • Add Windows specific code to avoid issues with including Windows.h.
  • Port storage.cc to C++.
  • Port some functions from storage.cc to use ghc::filesystem internally.
  • General cleanup related to the changes.

Some additional things I did that could use feedback:

  • Add USE_CPP_FILESYSTEM compile option to storage.cc, to keep legacy code around for testing.
  • Add blender::bli::filesystem as alias for ghc::filesystem. That should make the ghc -> std transition easier later on since we can just change this alias.
  • We could also introduce a way to switch between std::filesystem and ghc::filesystem at compile time.

Diff Detail

Repository
rB Blender
Branch
temp-cpp-file-system
Build Status
Buildable 16196
Build 16196: arc lint + arc unit

Event Timeline

Julian Eisel (Severin) requested review of this revision.Aug 3 2021, 6:32 PM
Julian Eisel (Severin) created this revision.
Julian Eisel (Severin) edited the summary of this revision. (Show Details)Aug 3 2021, 6:40 PM
Julian Eisel (Severin) added a project: Core.
Julian Eisel (Severin) edited the summary of this revision. (Show Details)Aug 3 2021, 6:52 PM
Julian Eisel (Severin) edited the summary of this revision. (Show Details)
Julian Eisel (Severin) edited the summary of this revision. (Show Details)
  • Remove wrong changes
  • Fix compile errors on Windows

I've seen namespace fs = std::filesystem in many examples. On one hand fs is a really short name, which we should avoid. On the other hand, there are very few other meanings of "fs" that I can think of (the other common one being "full screen"). I must say that in this case I personally prefer shorter code, like fs::path or fs::is_directory(file, error), over the longer one, filesystem::path or filesystem::is_directory(file, error). Nothing strong, just a slight preference.

I've seen namespace fs = std::filesystem in many examples. On one hand fs is a really short name, which we should avoid. On the other hand, there are very few other meanings of "fs" that I can think of (the other common one being "full screen"). I must say that in this case I personally prefer shorter code, like fs::path or fs::is_directory(file, error), over the longer one, filesystem::path or filesystem::is_directory(file, error). Nothing strong, just a slight preference.

I have mixed feelings about this. On one side I agree, it's annoying to have to write filesystem all the time. Then again I think, common Julian, it's not that hard of a word to write ;) And it actually makes code less cryptic I think.
We don't need to forbid using fs, but we shouldn't define that in a header at least. So I'd keep the namespace filesystem = ghc::filesystem in BLI_filesystem.hh, but if source files want to use fs, they are free to define that. Personally I'd still prefer not doing that, and using the full filesystem instead though.

source/blender/blenlib/BLI_fileops.h
50–52

Previously BLI_exists() was used to get the file mode. I split this into a separate function. Note that the file mode is accessed a bit differently for std::filesystem so we should basically get rid of usages of it.

source/blender/sequencer/intern/image_cache.c
427

The BLI_exists() check was redundant (BLI_is_dir() implies it).

Then again I think, common Julian, it's not that hard of a word to write ;) And it actually makes code less cryptic I think.

Ok, let's stand with filesystem and let the people who do have strong feelings about fs speak up if they don't agree.

  • Add USE_CPP_FILESYSTEM compile option to storage.cc, to keep legacy code around for testing.

To me it doesn't sound temporary enough, and removal of that #define from the code will go back to the current (unwanted) situation.
Maybe USE_LEGACY_FILESYSTEM or USE_OLD_C_FILESYSTEM_LIB? Then we can just remove the entire #define and it'll switch to the future.

  • Add blender::bli::filesystem as alias for ghc::filesystem. That should make the ghc -> std transition easier later on since we can just change this alias.

LGTM.

  • We could also introduce a way to switch between std::filesystem and ghc::filesystem at compile time.

This could/should be done with a CMake variable.

Maybe USE_LEGACY_FILESYSTEM or USE_OLD_C_FILESYSTEM_LIB? Then we can just remove the entire #define and it'll switch to the future.

+1.
We may have to move this to a higher level, so multiple source files can use it (everything that could be replaced by std::filesystem). So maybe makes sense as a CMake option even. No strong opinion.

  • We could also introduce a way to switch between std::filesystem and ghc::filesystem at compile time.

This could/should be done with a CMake variable.

Yeah definitely. Just wondering if we even want to have that option.

We may have to move this to a higher level, so multiple source files can use it (everything that could be replaced by std::filesystem). So maybe makes sense as a CMake option even. No strong opinion.

I was thinking the same thing. Could be nice to do some test builds just for the transitional period to see neither code paths are broken.

  • We could also introduce a way to switch between std::filesystem and ghc::filesystem at compile time.

This could/should be done with a CMake variable.

Yeah definitely. Just wondering if we even want to have that option.

In that case, let's add it when we know we need it.

I tested this patch on CentOS7, and it also works fine on that platform.

Julian Eisel (Severin) planned changes to this revision.Aug 11 2021, 4:46 PM
Julian Eisel (Severin) removed a project: Core.

I split off the part to introduce ghc::filesystem: D12197: Dependencies: Drop-in replacement for std::filesystem (in extern/).

Think I'll keep this open with "Changes Planned" for now.

Julian Eisel (Severin) abandoned this revision.EditedOct 12 2021, 12:04 PM

Abandoning this too, see D12197#326996.