Page MenuHome

Assets: add Asset Catalog system
ClosedPublic

Authored by Sybren A. Stüvel (sybren) on Sep 21 2021, 7:14 PM.

Details

Summary

Catalogs work like directories on disk (without hard-/symlinks), in that
an asset is only contained in one catalog.

See T90066 for design considerations.

Known Limitations

Only a single catalog definition file (CDF), is supported, at
${ASSET_LIBRARY_ROOT}/blender_assets.cats.txt. In the future this is
to be expanded to support arbitrary CDFs (like one per blend file, one
per subdirectory, etc.).

The current implementation is based on the asset browser, which in
practice means that the asset browser owns the AssetCatalogService
instance for the selected asset library. In the future these instances
will be accessible via a less UI-bound asset system.

The UI is still very rudimentary, only showing the catalog ID for the
currently selected asset. Most notably, the loaded catalogs are not
shown yet. The UI is being implemented and will be merged soon.

Catalog Identifiers

Catalogs are internally identified by UUID. In older designs this was a
human-readable name, which has the problem that it has to be kept in
sync with its semantics (so when renaming a catalog from X to Y, the
UUID can be kept the same).

Since UUIDs don't communicate any human-readable information, the
mapping from catalog UUID to its path (stored in the Catalog Definition
File, CDF) is critical for understanding which asset is stored in which
human-readable catalog. To make this less critical, and to allow manual
data reconstruction after a CDF is lost/corrupted, each catalog also has
a "simple name" that's stored along with the UUID. This is also stored
on each asset, next to the catalog UUID.

Writing to Disk

Before saving asset catalogs to disk, the to-be-overwritten file gets
inspected. Any new catalogs that are found thre are loaded to memory
before writing the catalogs back to disk:

  • Changed catalog path: in-memory data wins
  • Catalogs deleted on disk: they are recreated based on in-memory data
  • Catalogs deleted in memory: deleted on disk as well
  • New catalogs on disk: are loaded and thus survive the overwriting
Tree Design

This implements the initial tree structure to load catalogs into. See
T90608, and the basic design in T90066.

Diff Detail

Repository
rB Blender
Branch
master
Build Status
Buildable 17206
Build 17206: arc lint + arc unit

Event Timeline

Sybren A. Stüvel (sybren) requested review of this revision.Sep 21 2021, 7:14 PM
Sybren A. Stüvel (sybren) created this revision.
  • Asset Catalogs: include BLI_winstuff.h on Windows
  • Asset Catalogs: include BLI_winstuff.h on Windows
  • UUIDs: rename UUID to bUUID
  • Asset Catalog Test: reinstate commented-out cleanup code
  • Asset Catalog Test: avoid copying a file to a directory
  • Merge remote-tracking branch 'origin/master' into temp-asset-browser-catalogs
  • Cleanup: asset catalogs, avoid shadowing field by function parameter
  • Asset Catalogs: update explanatory comment in catalog definition file
  • Asset Catalog Tests: pass directory path instead of file path
  • Asset Catalog Tests: better path separator handling
  • Fileops: call BLI_path_slash_native() in BLI_dir_create_recursive(). This change is reviewed separately in D12611 and will be committed separately as well. It's just necessary to get everything in this patch working on Windows.
  • Cleanup: asset catalogs, explicitly name parameters
  • Asset Catalogs: return success from AssetCatalogService::write_to_disk
Julian Eisel (Severin) accepted this revision.EditedSep 23 2021, 2:55 PM

Didn't do much of a review, but I've worked with the code for a while now (some of it is even mine so I'm also biased :) ) and I think we're ready to do further development in master with this.

This revision is now accepted and ready to land.Sep 23 2021, 2:55 PM
This revision was automatically updated to reflect the committed changes.