Page MenuHome

Asset Browser: Internal File Browser changes & Python API additions
ClosedPublic

Authored by Julian Eisel (Severin) on Dec 2 2020, 10:18 PM.

Details

Summary

The Asset Browser will be a sub-editor of the File Browser. This prepares the File Browser code for that.

File-Lists

  • Support loading assets with metadata read from external files into the file-list.
  • New main based file-list type, for the "Current File" asset repository.
  • Refresh file-list when switching between browse modes or asset repositories.
  • Support empty file-lists (repository with no assets).
  • Store file previews as icons, so scripts can reference them via icon-id. See D9719.

Space Data

  • Introduce "browse mode" to differeniate between file and asset browsing.
  • Add FileAssetSelectParams to SpaceFile, with FileSelectParams as base. Makes sure data is separated between asset and file browsing when switching between them. The active params can be obtained through ED_fileselect_get_active_params().
  • FileAssetSelectParams stores the currently visible repository ID.
  • Introduce file history abstraction so file and asset browsing can keep a separate history (previous and next directories).

General

  • Option to only show asset data-blocks while file browsing (not exposed here).
  • Add "active_file" context member, so scripts can get and display info about the active file.
  • Add "active_id" context member, so ED_OT_lib_id_load_custom_preview can set a custom ID preview. (Only for "Current File" repository)
  • Expose some of FileDirEntry in RNA as (non-editable). That way scripts can obtain name, preview icon and asset-data.

Diff Detail

Repository
rB Blender
Branch
asset-filelists (branched from master)
Build Status
Buildable 11530
Build 11530: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
source/blender/editors/space_file/file_draw.c
41

This does not seem to be needed at all?

source/blender/editors/space_file/filelist.c
214–215

Aren't you missing to set history->browse_mode here?

226–233

Hrrrrrm...
BIP FLASH BIP
ACCESS AFTER FREE
;)

276

session UUID

277

id_session_uuid

333

Why?

394

Am a bit concerned about this storage of ID pointers, did you ensure this get cleared/rebuilt after any undo/redo? Not to mention plain basic file load (when not loading UI)?

724–767

Some more changes that should be committed separately as cleanups.

1045

I'd rather have an explicit proper comparison of members of asset_repository that should match, in a util function. Assuming a memory comparison is OK here is weak, to say the least.

1413

!= 0 is enough here, and is convention we use all across our codebase currently (except when checked value is compound bitflags).

1771–1774

Any reason not to call filelist_clear_ex(filelist, false, false, true); above instead?

1842

filelist->asset_repository != NULL

2025

MAIN_ID_SESSION_UUID_UNSET

2025

filelist->id_map == NULL

3226

Am confused, I though 'unset' value was -1 now?

source/blender/editors/space_file/filelist.h
69

Why can't you simply use FILE_TYPE_ASSET from eFileSel_File_Types instead?

source/blender/editors/space_file/filesel.c
423

!"Nice message explaining the assert reason." instead of just 0.

438–442

Same as above.

source/blender/makesdna/DNA_space_types.h
679

Can we keep ID for IDs in Blender? Could be UID here e.g.

686

same here, either just name, or uid_name?

782–789

Steel one from existing char _pad0[6] instead!

793

Thinks this needs an extra void *_pad1_v; or something? Remember safe way to deal with pointers in DNA is to always have an even of consecutive ones...

986

Why?

1075

Why is this changed to 64bits?

1087

id_session_uuid

1122

We try to avoid magic values like that, please use rather a define like #define FILEDIR_NBR_ENTRIES_UNSET -1 e.g.

And then check against that define, instead of the >= 0 or == -1 above in code.

This revision now requires changes to proceed.Dec 9 2020, 3:07 PM

Thanks! Will try to update ASAP. Looking through this again in more detail, I realize some things seem a bit rushed...

source/blender/editors/space_file/filelist.c
214–215

Looks like it, good catch! This worked very well in my tests so I'm surprised in hindsight.

226–233

Yeah, noticed and fixed that yesterday. I swear, check the commit date on the next patch update! :D

333

Quick code archeology... Ah, seems it's not needed anymore, I used to store the FILE_TYPE_ASSET as (1 << 32) but that changed.

394

No I didn't! But it's the main blocking TODO for the merge: T83242: Ensure "Current File" repository stays in sync with main data-base.

(Also, see my first comment here.)

1413

It is a compound one actually: FILE_TYPE_ASSET | (1 << 29). But I don't like that and will change it.

1771–1774

Don't think so, looks like an oversight.

3226

It's set to 0 in filelist_readjob_startjob(), but indeed, that's confusing. Should probably just set it to 0 here to declare it as a valid, empty list.

source/blender/makesdna/DNA_space_types.h
782–789

Hmm, but I stole one from _pad1. Isn't that better since _pad0 is part of the SpaceLink base?
Looking at write/read code, we write the data with the correct SDNA ID (ie not SpaceLink) so reconstruction should be fine, even if the _pad0 is used. But still looks like a potential source of problems.

986

Same as above I think, because I used 1 << 32 in the beginning which would cause warnings otherwise.

Julian Eisel (Severin) marked 22 inline comments as done.Dec 10 2020, 12:45 AM
Julian Eisel (Severin) added inline comments.
source/blender/editors/space_file/filelist.h
69

I'm not sure what you mean. That I don't just set FILE_TYPE_ASSET for the filter parameter? That wouldn't work since the filters are effectively OR'ed together, but for "Only Assets" we want to AND it to the other filter options, if that makes sense?

Or do you mean adding a new parameter taking the file types to limit visibility to?

source/blender/editors/space_file/filesel.c
423

Even better, removed the assert and the return. Since we cast to eFileBrowse_Mode above, the compiler is smart enough to see that this point can not be reached, and won't compile if a case is not handled. (Think usually if a enum case isn't handled you just get a warning?)

source/blender/makesdna/DNA_space_types.h
686

Hmm, may I disagree? :) For me name is something shown in the UI, but isn't necessarily unique. If it's a unique identifier -> idname. Grepping over the code - we use idname quite a lot, and it's obvious what it is. In the cases where we use just name, I'm not sure if this is an identifier or just a UI name. Didn't find any usages of uid_name.

793

We don't seem to use the _v suffix anywhere else, so going with void *_pad2.

Julian Eisel (Severin) marked 2 inline comments as done.
  • Avoid use-after-free
  • Address review comments
  • Fix "Only Assets" filter hiding non-data-block file types
  • Repositories are now called "Asset Libraries"

After checking with Brecht in D9722 and discussion with the UI team, we'd like
to use that term instead.

source/blender/editors/space_file/filesel.c
423

Hrmm, seems GCC doesn't like that idea. Will have to bring the return back.

Bastien Montagne (mont29) requested changes to this revision.Dec 10 2020, 10:56 AM

T83242 is indeed a blocking TODO for this patch...

source/blender/editors/space_file/filelist.c
2002

Errrrr, really? Would expect <= here? Or even better, ==.

2034

Again, ==

source/blender/editors/space_file/filelist.h
69

filter is supposed to contain eFileSel_File_Types bitflags right? then it should already contain the FILE_TYPE_ASSET one... No need to explicitly pass that one as an extra parameter here?

source/blender/editors/space_file/filesel.c
423

GCC can check that you do handle all defined enums in your case's clauses, but it cannot ensure that you actually pass a valid value as browse_mode, so you still need a default return value indeed.

source/blender/makesdna/DNA_space_types.h
686

Fact that we do a lot of bad things in existing code is not exactly a valid reason to keep doing it in new one, especially if it can be easily avoided.

Once again, I'd rather keep id for actual ID stuff, and use of uid when we mean identifier sounds reasonable to me.

Or you could also use full identifier (as we already do in areas like RNA or depsgraph e.g.), instead of idname...

782–789

Oh, you are totally right here, my bad.

This revision now requires changes to proceed.Dec 10 2020, 10:56 AM
Julian Eisel (Severin) marked 8 inline comments as done.
  • Fix invalid memory use of "Current File" after undo & deleting data-blocks
  • Refactor how we store ID and asset data references in file lists
  • Don't run thread-unsafe file-list reading in threads
  • Address review comments
  • Avoid unsafe access to asset browsing through BPY

With this updated there are no blocking TODOs from my side.

Here's the idea for keeping ID references in the file-list intact:

  • File reading (including undo, loading without UI or linking screens) sets the FILE_REBUILD_MAIN_FILES to SpaceFile.rebuild_flag. Before drawing this is checked and the file-list reset if necessary.
  • ID deletion is dealt with via the SpaceType.id_remap callback. Rather than remapping, we just force the file-list to reset. We could add a filelist_id_remap() function to avoid a full reset, but that's a further improvement we can do.
  • Removing the asset metadata (Clear Asset) must send a NC_ASSET notifier. The file-list can be reset based on that.

The session UUID usage is removed now, I don't see any benefits over using ID pointers directly.

source/blender/editors/space_file/filelist.c
266

Ak, why is that still there? Removed locally now.

2002

WTH... what was I doing there?

source/blender/editors/space_file/filelist.h
69

But file types passed via filter are OR'ed together. That's not what we want for FILE_ASSETS_ONLY, there we need a logical AND. It should remove all files that are *not* FILE_TYPE_ASSET, however filter keeps files that *are* of the given types.

So filter includes, filter_assets_only excludes files.
We could of course just make the filtering behave differently for that file type filter but, eeeh...

source/blender/editors/space_file/filesel.c
423

Well, the surprising thing is that Clang did *not* complain. This is clearly a possible source of bugs, as GCC correctly complains about.

Bastien Montagne (mont29) requested changes to this revision.Dec 11 2020, 12:29 PM

Think we are pretty close to be done here...

source/blender/editors/space_file/filelist.c
278

The Space is responsible to take care of that.

414

tags

427

FILELIST_TAGS_...

2020–2026

Do we still need this? Am not a big fan of 'accessors' that just return the property itself...

source/blender/editors/space_file/filelist.h
69

Oh, OK, was sort of confusing the FILE_TYPE_ASSET and the FILE_ASSETS_ONLY here...

Think we should just pass params here, instead of reading all kind of things from it in caller code, would make things cleaner, easier to read, and easier to extend.

Ideally this refactor could be done before this gets merged, but if you think it's easier afterward it's fine too, as long as we do not forget to do it. ;)

source/blender/makesdna/DNA_space_types.h
788–789

Runtime flags are called tags by convention in Blender.

Also, think we could just call them tags, make them a short, that way it can be used in the future for other purposes as well?

890

FILE_TAG_...

This revision now requires changes to proceed.Dec 11 2020, 12:29 PM
Julian Eisel (Severin) marked 8 inline comments as done.
  • Undo unnecessary int type change
  • Fix crash when splitting area with invisible file browser data
  • Cleanup: Address review comments
source/blender/editors/space_file/filelist.c
2020–2026

We don't really need this, but most members of FileDirEntry are already accessed via filelist.c only. Think with just 2 or 3 more of these dumb accessors, we can move the struct out of DNA into filelist.c. I find that a good idea, but will do whatever you prefer.

source/blender/editors/space_file/filelist.h
69

Added a task to the workboard: T83656: Code cleanup

OK, think it's fine for master now.

source/blender/editors/space_file/filelist.c
2020–2026

OK fine then

source/blender/makesdna/DNA_space_types.h
886

tags

This revision is now accepted and ready to land.Dec 11 2020, 2:27 PM