Page MenuHome

Asset Browser: Milestone 1 - Basic, Local Asset Browser
AbandonedPublic

Authored by Julian Eisel (Severin) on Nov 23 2020, 7:45 PM.

Details

Summary
NOTE: This is not ready for a merge yet. Purpose of this patch is to give reviewers a first look. I plan to address some TODOs marked in code, and then merge it soon to continue work in master. The patch can go through some proper review then.

For remaining TODOs and project description, see Asset Browser (Archived) (Milestone 1: Basic, Local Asset Browser).

This patch includes:

  • "Make Asset" operator, exposed in Outliner and button context menus (the latter is disabled currently)
  • Asset meta-data storage and management
  • Custom asset tags
  • Reading asset meta-data for separate files (but not more)
  • Automatic object previews
  • Custmo Asset repository setup via Preferences (has to be .blend files currently), directories are a ToDo.
  • Default repository at ~/assets.blend on Linux/macOS, ~/Documents/assets.blend on Windows
  • Asset Browser editor, internally a sub-editor of the File Browser
  • Internal File Browser changes need for asset browsing
  • "Current File" repository for the File Browser
  • Basic UI for editing asset meta-data
  • Dragging object, collection, image and material assets into 3D Views
  • Option to only show assets on Append/Link
  • Some unfinished code for per repository Asset Catalogs (should likely be disabled for now)

Asset Browser UI:

Demo (a bit old, UI changed slightly since then):Threaded generation of object previews:

Repository UI in Preferences:

Diff Detail

Repository
rB Blender
Branch
asset-browser
Build Status
Buildable 11492
Build 11492: arc lint + arc unit

Event Timeline

Julian Eisel (Severin) requested review of this revision.Nov 23 2020, 7:45 PM
Julian Eisel (Severin) created this revision.
Julian Eisel (Severin) planned changes to this revision.Nov 23 2020, 7:48 PM
release/scripts/startup/bl_ui/space_userpref.py
1269–1274

This could become a UIList, but I kept it consistent to the "Excluded Paths" UI for the autorun scripts.

source/blender/blenkernel/BKE_asset.h
36–41

This is for per repository data, like repository-level catalogs (e.g. to group together poses as a "library").
WITH_ASSET_REPO_INFO should be disabled for now and relating structs be disabled for SDNA.

(The current design for this is to write a single asset repository data block to a file as a new global struct, so that it can be read without reading the entire file. Similar to REND. The meta-data of a repository is then the union of all these global repository data blocks from all the .blends. --- Obviously this design needs discussion.)

source/blender/blenloader/intern/readblenentry.c
163–164

Comment needs updating.

source/blender/editors/asset/asset_edit.c
55

Should maybe set (or ensure?) a real user.

source/blender/editors/asset/asset_ops.c
39–66

I don't think we should do this, too expensive for a poll(). Was just experimenting with it.
I'd love to have grayed out buttons and a disabled-hint if needed in context menus based on selection, rather than always having the operators there and having them fail.

source/blender/editors/interface/interface_context_menu.c
969

This is currently not working. T82664

source/blender/editors/space_file/filelist.c
392–394

Would be good to have done before a release, but can be a bcon2 task.

1706

This is a showstopper for me, I don't want to cause thread-races.
Generating the file-list from Main should be quite fast, we could just disable threading or block for this.

2850–2872

This is a bit messy, would like to clean it up.

source/blender/makesdna/DNA_ID.h
271–272

Outdated comment, should remove it.

source/blender/makesdna/DNA_asset_types.h
60–61

There's not much point in this, should always use the ID's preview.

67–74

I'm not sure if we should keep these properties here. They could just be added as custom properties if needed. At least the description and author fields.

source/blender/windowmanager/intern/wm_operator_type.c
280 ↗(On Diff #31310)

Not needed anymore.

There's some places where I think usability needs to be improved, or that I think need to be more complete before exposing this to users. But I'll wait with that feedback since it's not entirely clear what is planned to be completed still.

Mainly reviewing the data structures and naming here.

release/scripts/startup/bl_ui/space_filebrowser.py
572

We use Metadata without the hyphen in other places in the UI like image editor and sequencer.

604

We could consider having this simply as a text field separated by spaces, since it's faster to add or copy-paste a bunch of tags, and commonly done that way on the web.

That can be done regardless of how the data is stored internally though.

The lack of multiline textbox for this and description is unfortunate, would be good to support that at some point in the future.

release/scripts/startup/bl_ui/space_userpref.py
1258

Perhaps put this panel at the top or bottom, it looks a bit off to me to have this between other panels that all look similar.

1272–1273

I'd suggest to switch the order and put name first.

1277

Suggest to align this on the right side.

source/blender/editors/space_file/filelist.c
3247

Are we sure this pointer remains valid, that the ID or preview does not get deleted?

In general, what is the mechanism to ensure these entries stay in sync with the main database?

source/blender/makesdna/DNA_asset_types.h
32

I suggest to name this AssetTag or MetaDataTag. I'm not sure why "custom" is needed, aren't all tags custom?

59

Personally would name this AssetMetaData, since "data" doesn't mean much.

60–61

Agree, should be removed.

67–74

I think it makes sense for description and tags to be built-in. For everything else I guess we use ID properties presented in a nice UI?

Something like custom properties, but a bit nicer and tuned to this use case, and eventually some options to add standard metadata types like author or license.

source/blender/makesdna/DNA_space_types.h
679

I'm not sure the struct needs to be this specific. Would it make sense to have a single struct that store all asset browser specific settings, which includes this ID?

source/blender/makesrna/intern/rna_asset.c
207

I'm not sure we should have author as a native property, but if we do then this description seems wrong to me. It's the creator of the asset, responsibility is something else.

source/blender/windowmanager/WM_api.h
685–693

The naming here is getting a bit confusing to me.

Maybe it helps to rename existing functions and enum items to "local ID", and then use ID to refer to cases where it can be an ID from any source? Ideally the distinction between the two can be mostly abstracted for drop code.

I just tested briefly tested and looked over a bit of the code.

release/scripts/startup/bl_ui/space_filebrowser.py
47

I just tested this and it isn't true anymore, it also changed with my adjustments to the way this is exposed in the "non asset file browser" a few months ago. (Other uses of .popover( in this file).

51

Also, looks like the recursions property isn't initialized properly:

source/blender/blenkernel/intern/asset.c
107

This pointer was already used above, right? So no need for the "safe" part here, this function assumes it.

source/blender/blenloader/intern/readfile.c
970

Typo: and -> an

Julian Eisel (Severin) marked 3 inline comments as done.Nov 30 2020, 5:01 PM
Julian Eisel (Severin) added inline comments.
release/scripts/startup/bl_ui/space_filebrowser.py
47

Hmm, if I try this, and don't manually set the icon, I get this:

But what I want is this of course:

51

The UI here is kind of a dummy one and needs to be done properly. That's part of T82680.

604

I personally prefer the UIList layout for now. Communicates much better that you're actually adding separate items/tags, not just a string.
For me, a dedicated tag editing widget/template is part of the usability improvements milestone, including multiline support (see D7496). It could support copy/paste and automatically display separate tags as you enter or paste commas.

source/blender/blenkernel/intern/asset.c
107

I generally prefer MEM_SAFE_FREE(), not just because of the null-check but also because it nulls the pointer. Avoids nasty mistakes.
But actually I'll change this to a slightly different pattern.

source/blender/editors/space_file/filelist.c
3247

It's unsafe and I plan to address this, see T82886: Crash on undo while showing "Current File" repository. Should definitely be done before the merge.
Also created T83242: Ensure "Current File" repository stays in sync with main data-base, although it's kinda the same issue really.

It shouldn't store a reference to ID data like this (same for the asset-data). At least not without extra sanity checks. The preview will later be copied into an image buffer, but may break meanwhile because of the threading.

I already added FileListInternEntry.id_uuid and FileList.id_map, which we could use to sanity check the references before access.

source/blender/makesdna/DNA_space_types.h
679

We also store this in the file-list. That way we can check if the file-list needs refreshing. I generally like small, focused structs/classes, although in this case it also affects SDNA of course.

How would you imagine the single struct to look like, what would it contain?

source/blender/windowmanager/WM_api.h
685–693

Yeah agreed.
And once the drag/drop rewrite is in eventually, the distinction should indeed be abstracted away.

release/scripts/startup/bl_ui/space_filebrowser.py
47

Ah, I see the issue. Before my recent change to these popovers in the file browser I noticed that this gives the popover button some suboptimal tooltips, but that's a small issue. I can revisit this when the UI is further along I suppose : )

51

Okay, thanks. I wasn't sure what was changing in the future and what wasn't.

IMHO would really help, both for readability and avoiding too much different topics in comments, if that patch was split in smaller parts (one for UI changes, one for core datamanagement changes, one for filebrowser changes, etc.)? Right now it's kind of a monster patch... Cannot really see how this is expected to be reviewed and approved in one or two weeks?

Julian Eisel (Severin) updated this revision to Diff 31498.EditedDec 1 2020, 7:29 PM
Julian Eisel (Severin) marked 11 inline comments as done.
  • Support custom preview images for assets/IDs (copies buffer of selected image file to the ID's preview)
  • Proper implementation of "Make Asset" for button context menus (taking "focused_id" context member)
  • Turn file previews into icons to allow access from BPY
  • Show asset preview in Asset Browser sidebar
  • Compile BKE icons source file with C++ & use C++ scoped mutex locks
  • Remove preview pointer in asset-data
  • Rename some core asset types (from review)
  • Remove WIP code for asset catalogs
  • Minor UI tweaks (from review)
  • Remove "Author" from asset metadata (from review)
  • General cleanup (from review)

@Bastien Montagne (mont29) yes I expected I'd have to split it up, it seemed more important to have it up at all. Will do.

To be clear, work will continue after merge, the milestone is not completed. I'd like to work closer to master with smaller, more frequent reviews. This is also to get more testing and feedback. Similar to how the nodes guys are doing it. And not to forget, even with the current limited scope, this project adds urgently needed value for users. (I was actually pushing against merging as non-experimental.)
As far as I'm concerned, all we need to review here is the general design. If you'd like to see internal (non-compatibility-breaking) changes, I can add that to the backlog which I actively work on, and we can organize further reviews for after the merge if you want to. But I only see general design issues or compatibility-breaking requests as blockers that can justify further delays, if you see what I mean.

source/blender/makesdna/DNA_asset_types.h
67–74

I didn't expect we'd make the custom properties editable from the UI, but guess it's reasonable. If you think it's needed I will create a task for it.
Would you agree this is lower priority for now, something we can wait for until 2.93 even?

Removed the author field btw.

source/blender/windowmanager/WM_api.h
685–693

Changed the naming to be more clear and used "local ID".
For the last function I chose WM_drag_get_local_ID_or_import_from_asset(), think it's good to be clear.

Julian Eisel (Severin) abandoned this revision.EditedDec 2 2020, 11:08 PM

Split the patch into 7 parts now, check the list in T82819.
Closing this, but the joy goes on.