Hi @Richard Antalik (ISS) I would like you to join a small slack workspace with myself and a few others who are interested in upgrading the VSE.
If you can share with me an email address I would be happy to send an invite to you.

Hi, it's richardantalik at google mail

Great, thanks! invite sent.

I try to compile this, but the diff file no longer works (can't patch). Can anyone guide me? Is it for the master branch? Is there any chance that this can work in Blender 2.80?

In D3597#86749, @Jens Ruda (jaker) wrote:

I try to compile this, but the diff file no longer works (can't patch). Can anyone guide me? Is it for the master branch? Is there any chance that this can work in Blender 2.80?

Hi, my plan is to maintain this in my own branch, so once beta is out, I will rebase this.
I am too busy now so unfortunately there may be some delay.

if you want win32 binary, you can download it there https://drive.google.com/open?id=112jLW3yjXIP0RHMiw6K6GUwtinX3QYpA

@Richard Antalik (ISS)
I've been playing around with your 'test patch D3597'

A few remarks:

• The progress bar on the audio strip visually suggests that a proxy is being generated for the audio, even though it's not. (I would remove it)
• Overwrite does not seem to work. (When set to disabled)

I think you'd have to enable the panel for the clip strips, though I'm not sure if this is outside of you plan (supporting clip strip proxies with and without distortion in the VSE)

Feature request:

Please... please... can we have the option to render image sequences.

They can save so much time by just deleting the frames you need re-rendered and then rendering with overwrite disabled.

Misc:

This is probably an issue with Blender itself, but if you select a location that is read only Blender will crash.
Same if the drive is full.

In your build I can't seem to add Image sequence strips.

In D3597#89058, @Christopher_Anderssarian wrote:

@Richard Antalik (ISS)
I've been playing around with your 'test patch D3597'

• The progress bar on the audio strip visually suggests that a proxy is being generated for the audio, even though it's not. (I would remove it)

This would be easy to solve.

• Overwrite does not seem to work. (When set to disabled)

Will look at this.

I think you'd have to enable the panel for the clip strips, though I'm not sure if this is outside of you plan (supporting clip strip proxies with and without distortion in the VSE)

Did you build install target? I see file release/scripts/startup/bl_ui/space_sequencer.py included in patch.

Feature request:

Please... please... can we have the option to render image sequences.

They can save so much time by just deleting the frames you need re-rendered and then rendering with overwrite disabled.

Sure, we can pass ibufs to different encoding engine. Dunno if it's better to use older custom solution, or if ffmpeg can be configured to do such task.
As there are some TODO's still I would push this down the priority list.

Render mutex and invalidation may conflict with cache "remake" I am working on, therefore I want to wait for result of that patch before implementing.

Misc:

This is probably an issue with Blender itself, but if you select a location that is read only Blender will crash.
Same if the drive is full.

I am pretty sure, that blender has some functions built in to deal with this. It didn't come to my mind even that this would be an issue.

In your build I can't seem to add Image sequence strips.

Must look into this - it was not my intention.

Hi @Richard Antalik (ISS) , I'm trying to test your patch, but I am unable to apply it to the master branch. What is the base commit for this patch ?

Once I can build it, I think I can tidy this patch up a bit, remove warnings etc.

In D3597#96185, @Guillaume M (mathers) wrote:

Hi @Richard Antalik (ISS) , I'm trying to test your patch, but I am unable to apply it to the master branch. What is the base commit for this patch ?

Once I can build it, I think I can tidy this patch up a bit, remove warnings etc.

You would have to look by date, but this patch needs to be reworked:

• to keep things simple, code cleanup done in this patch should be dropped
• multithreading code should be rewritten in similar fashion to D3934. Also it should be consulted with other developper, with more experience with FFMPEG, probably even with FFMPEG developper. Approach used here worked however...
  • use one proxy job, as controller for sub jobs
  • work on scene copy
  • there should be lock on strips, that are being processed)

Only strips with video file input should be supported. Instead of using proxy system to handle all strip types, cache with disk dumping will be used.
reasons are:

• multiple processing steps in pipeline (proxies are ideally 1:1 copy, but optimized for scrubbing or scaled to increase performance)
• changes in timeline will affect cached result, sometimes partially
  • invalidation is easier using cache
  • image export instead of ffmpeg must be used

If you want to help me with this, I will be happy to help / guide you in process.

You would have to look by date, but this patch needs to be reworked:

• to keep things simple, code cleanup done in this patch should be dropped

You mean this patch should be re-written from scratch ?

• multithreading code should be rewritten in similar fashion to D3934. Also it should be consulted with other developper, with more experience with FFMPEG, probably even with FFMPEG developper. Approach used here worked however...
  • use one proxy job, as controller for sub jobs
  • work on scene copy
  • there should be lock on strips, that are being processed)

I don't think we should generate several proxies in parallel :

• when encoding a movie proxy, ffmpeg should already be multi-threaded when working on a single file (I'm not sure if it is implemented in blender, but it certainly works with ffmpeg cli)
• when encoding a non-vse scene proxy (3d renders), the rendering process is already multi-threaded

By the way, when generating several proxies (25%...100%) for a non-vse scene (3d render), the fastest way would be to generate the highest-res proxy first, then downscale it to obtain the other proxy resolutions, right ? How feasible would this be ?

Only strips with video file input should be supported. Instead of using proxy system to handle all strip types, cache with disk dumping will be used.
reasons are:

• multiple processing steps in pipeline (proxies are ideally 1:1 copy, but optimized for scrubbing or scaled to increase performance)
• changes in timeline will affect cached result, sometimes partially
  • invalidation is easier using cache
  • image export instead of ffmpeg must be used

I'm not sure what you mean by that. I think it's important to separate proxies and pre-fetching. Actually I'm still not sure why we are switching from image sequences to avi files for proxies, I feel image files were more flexible and allow for easier scrubbing and pre-fetching.

If you want to help me with this, I will be happy to help / guide you in process.

Yes I'd like to spend a bit of time on these issues, but I feel this patch is getting way too big and hard to merge/apply. I think it's important to separate proxy generation, pre-fetching and interface changes in several incremental patches (trying to keep their size to <100 lines).

I'm not sure how the blender git tree is organized, but from my perspective it would make sense to develop these features in a separate branch, and to make minimal (and documented) changes to the API, so it is easier to merge in the master branch later.

In D3597#96209, @Guillaume M (mathers) wrote:

You would have to look by date, but this patch needs to be reworked:

• to keep things simple, code cleanup done in this patch should be dropped

You mean this patch should be re-written from scratch ?

Contributor has to decide ultimately. I don't have means to force this :)

I don't think we should generate several proxies in parallel :

• when encoding a movie proxy, ffmpeg should already be multi-threaded when working on a single file (I'm not sure if it is implemented in blender, but it certainly works with ffmpeg cli)

Good - code will be cleaner. ultimately we should check if multithreading works properly.

• when encoding a non-vse scene proxy (3d renders), the rendering process is already multi-threaded

By the way, when generating several proxies (25%...100%) for a non-vse scene (3d render), the fastest way would be to generate the highest-res proxy first, then downscale it to obtain the other proxy resolutions, right ? How feasible would this be ?

Not applicable - as I said this will be done by cache management. *Only* movies will have proxies.

Only strips with video file input should be supported. Instead of using proxy system to handle all strip types, cache with disk dumping will be used.
reasons are:

• multiple processing steps in pipeline (proxies are ideally 1:1 copy, but optimized for scrubbing or scaled to increase performance)
• changes in timeline will affect cached result, sometimes partially
  • invalidation is easier using cache
  • image export instead of ffmpeg must be used

I'm not sure what you mean by that. I think it's important to separate proxies and pre-fetching.

First let's clear definitions: Proxy is close enough representation of original file, pre-processed to have required properties(fast scrubbing, scaled), so working with it is more time efficient.

Prefetching has nothing to do with dumping cache.
As cache is generated, it can be stored on disk. When cache is polled for frame, it will look first in RAM, if frame is not found, is can look at disk - still faster then re-render in most cases.
this dumped data however *have to be* stored individually, as it may be requested to be deleted/flagged as out-of-date (properties of strip has changed -> invalidate cache).

Actually I'm still not sure why we are switching from image sequences to avi files for proxies, I feel image files were more flexible and allow for easier scrubbing and pre-fetching.

FFMPEG recoding is supposed to be faster then saving images.
As for speed of loading data, both methods have very similar performance.

If you want to help me with this, I will be happy to help / guide you in process.

Yes I'd like to spend a bit of time on these issues, but I feel this patch is getting way too big and hard to merge/apply. I think it's important to separate proxy generation, pre-fetching and interface changes in several incremental patches (trying to keep their size to <100 lines).

I'm not sure how the blender git tree is organized, but from my perspective it would make sense to develop these features in a separate branch, and to make minimal (and documented) changes to the API, so it is easier to merge in the master branch later.

If FFMPEG is keeping CPU busy even with 1 thread, there is nothing much to be done performancewise.

Some cleaning of old code, refactoring and updating UI to reflect reality
Small changes are good approach, but end goal should be discussed. With prefetching I changed same code number of times, because I was not familiar with basics, and therefore couldn't plan changes in detail.

By the way, when generating several proxies (25%...100%) for a non-vse scene (3d render), the fastest way would be to generate the highest-res proxy first, then downscale it to obtain the other proxy resolutions, right ? How feasible would this be ?

Not applicable - as I said this will be done by cache management. *Only* movies will have proxies.

I'm not sure what you mean by that. I think it's important to separate proxies and pre-fetching.

First let's clear definitions: Proxy is close enough representation of original file, pre-processed to have required properties(fast scrubbing, scaled), so working with it is more time efficient.

Prefetching has nothing to do with dumping cache.
As cache is generated, it can be stored on disk. When cache is polled for frame, it will look first in RAM, if frame is not found, is can look at disk - still faster then re-render in most cases.
this dumped data however *have to be* stored individually, as it may be requested to be deleted/flagged as out-of-date (properties of strip has changed -> invalidate cache).

Yes I'd like to spend a bit of time on these issues, but I feel this patch is getting way too big and hard to merge/apply. I think it's important to separate proxy generation, pre-fetching and interface changes in several incremental patches (trying to keep their size to <100 lines).

I'm not sure how the blender git tree is organized, but from my perspective it would make sense to develop these features in a separate branch, and to make minimal (and documented) changes to the API, so it is easier to merge in the master branch later.

Some cleaning of old code, refactoring and updating UI to reflect reality
Small changes are good approach, but end goal should be discussed. With prefetching I changed same code number of times, because I was not familiar with basics, and therefore couldn't plan changes in detail.

If I understand correctly, the plan is to :

• Drop proxy support for all sequences except movies (it would also be dropped for clips and image sequences right ?). All proxies would be ffmpeg-encoded avi files, no image sequences.
• Improve the RAM-cache already present in blender with a second-level disk-cache for all sequences

If that is correct, I think this should be split in two patches, one focusing on proxies, and a second one focusing on implementing disk cache (this should probably be in a new file ?)

I also think the title of this diff is pretty misleading since it actually removes proxy support for all sequences (except movies) and adds disk cache.

Since proxies are only for movies, I think ot would make sense to add a "batch process" feature so the user can generate proxies for all his footage before even importing them in blender. I find it frustrating to wait for proxy generation each time I import a clip.

The reason I mention this now is because this would require knowing in advance where the proxy files should be generated, but for now it is linked to each clip.

Should we make this "proxy path" a configuration a project-wide param ? Do we need a per-clip override ?

If I understand correctly, the plan is to :

• Drop proxy support for all sequences except movies (it would also be dropped for clips and image sequences right ?). All proxies would be ffmpeg-encoded avi files, no image sequences.

Not quite. Movieclips do have "own" proxy management. Builder is shared, but you can not build proxies for movieclip from sequencer

Images are already loaded fast. If scaled proxy of image has to be used, we can smaller image. FFMPEG builder can be used, it would make consistent naming scheme, but other than that there are only disadvantages.

As I said, image sequences are arguable...

• Improve the RAM-cache already present in blender with a second-level disk-cache for all sequences

If that is correct, I think this should be split in two patches, one focusing on proxies, and a second one focusing on implementing disk cache (this should probably be in a new file ?)

FFMPEG builder can be found in imbuf module file indexer.c. In sequencer proxy building is fragmented and all over sequencer code. this patch did clean it up, however since both sequencer and movieclip is using this API this turned out to be a little bit tricky.

Cache dumping should be in imbuf module in 'moviecache.c' as this cache is "universal" cache for storing images and it's used across blender. This way not only sequencer, but other projects can benefit also.

I also think the title of this diff is pretty misleading since it actually removes proxy support for all sequences (except movies) and adds disk cache.

Maybe usable, but very impractical disk cache...
This patch was done a lot earlier, then I was thinking about cache...

In D3597#96252, @Guillaume M (mathers) wrote:

Since proxies are only for movies, I think ot would make sense to add a "batch process" feature so the user can generate proxies for all his footage before even importing them in blender. I find it frustrating to wait for proxy generation each time I import a clip.

How would you implement that?
There are scripts, addons and a lot of info how to create proxies outside(or even inside using non-standard methods) of Blender

The reason I mention this now is because this would require knowing in advance where the proxy files should be generated, but for now it is linked to each clip.

Should we make this "proxy path" a configuration a project-wide param ? Do we need a per-clip override ?

You can set 3 settings:

• path per strip
  • Per strip is good option, if you want to move / archive data with proxies pre-built, or multiple projects are using same video files - no need to generate proxy per project
• path per project
  • Per project setting is good, if you want to have only proxies groupped at one place.
• mannual override
  • mannual override is necessary if you have 2 instances of one sequence, and for some reason you need to change proxy for another file, while other instance must use normal proxy file.

I don't see these settings as a complication for development or anything

In D3597#96296, @Richard Antalik (ISS) wrote:

If I understand correctly, the plan is to :

• Drop proxy support for all sequences except movies (it would also be dropped for clips and image sequences right ?). All proxies would be ffmpeg-encoded avi files, no image sequences.

Not quite. Movieclips do have "own" proxy management. Builder is shared, but you can not build proxies for movieclip from sequencer

I see, so dropping proxies for all sequences except movie and movieclip

Images are already loaded fast. If scaled proxy of image has to be used, we can smaller image. FFMPEG builder can be used, it would make consistent naming scheme, but other than that there are only disadvantages.

Ok, let's say no image sequence proxies for now

• Improve the RAM-cache already present in blender with a second-level disk-cache for all sequences

If that is correct, I think this should be split in two patches, one focusing on proxies, and a second one focusing on implementing disk cache (this should probably be in a new file ?)

FFMPEG builder can be found in imbuf module file indexer.c. In sequencer proxy building is fragmented and all over sequencer code. this patch did clean it up, however since both sequencer and movieclip is using this API this turned out to be a little bit tricky.

I'm not sure why we need to change this API ? I'll take a deeper look at your patch.

Cache dumping should be in imbuf module in 'moviecache.c' as this cache is "universal" cache for storing images and it's used across blender. This way not only sequencer, but other projects can benefit also.

Sure. I suppose we should use image sequences for this cache ?

I also think the title of this diff is pretty misleading since it actually removes proxy support for all sequences (except movies) and adds disk cache.

Maybe usable, but very impractical disk cache...
This patch was done a lot earlier, then I was thinking about cache...

There are scripts, addons and a lot of info how to create proxies outside(or even inside using non-standard methods) of Blender

In the long run, I think "asset preparation" should be an workspace of its own for the VSE, but let's keep this as a future project.

The reason I mention this now is because this would require knowing in advance where the proxy files should be generated, but for now it is linked to each clip.

Ok, I hadn't realized yet that proxy_dir was shared across all strips

Generally I find the proxy controls are extremely confusing in blender, especially for beginners :

• In the VSE
  • The "proxy storage" option (project or strip) is a project-wide option, but it is available only when selecting a strip. It should be moved to user preferences, or left in the N-menu with its own header, always visible (same as "Annotations" is now)
  • The "proxy custom directory" and "proxy custom file" options for "per strip" proxies are confusing : is blender going to overwrite the file, or just going to read it ?
  • "Set Selected Strip Proxies" and "Rebuild Proxy and Timecode Indices" should be moved to a global toolbar menu, same as "proxy storage"

• In the movie clip editor
  • It's not obvious that "proxy size" is related to the current display, and not the proxy building process

If that's ok with you I'll start cleaning up the code in small incremental patches, starting from the current master branch, 25772c9e1d2d5239e1f7f83f6ea0c4d7d1f1df4a

In D3597#96364, @Guillaume M (mathers) wrote:

I'm not sure why we need to change this API ? I'll take a deeper look at your patch.

What can be done to proxies now is

• cleanup in sequencer part
• PNG codec support was not a bad idea. It has a bit worse performance, but it has transparency.
• Even more demanding lossless codec can be implemented - FFv1, huffyuv...
• it may be useful to introduce scale to specific size - VSE should support this as-is I think. Also See last section.

Cache dumping should be in imbuf module in 'moviecache.c' as this cache is "universal" cache for storing images and it's used across blender. This way not only sequencer, but other projects can benefit also.

Sure. I suppose we should use image sequences for this cache ?

Yes.

I also think the title of this diff is pretty misleading since it actually removes proxy support for all sequences (except movies) and adds disk cache.

Maybe usable, but very impractical disk cache...
This patch was done a lot earlier, then I was thinking about cache...

There are scripts, addons and a lot of info how to create proxies outside(or even inside using non-standard methods) of Blender

In the long run, I think "asset preparation" should be an workspace of its own for the VSE, but let's keep this as a future project.

Not sure, what do you mean by "asset preparation", but it sounds scary :)

Generally I find the proxy controls are extremely confusing in blender, especially for beginners :

Yes, UI is not the greatest. it is being worked on, but there are hotter issues. Mainly this prefetching... It should have be done by now, so it can be tested

• In the movie clip editor
  • It's not obvious that "proxy size" is related to the current display, and not the proxy building process

When you choose size of 50% this will reduce movie resolution by 50%, then during rendering proxy is scaled up by 50%
This can be annoying, when you have files with more than one resolution.
For example When I use (""slow" motion") 320 x 240 movie along with FHD movie, make proxies for all movies, I end up with 160 x 120 file...
I think, that most users would want to have all proxies of the same resolution, as that is much closer to final render.

If that's ok with you I'll start cleaning up the code in small incremental patches, starting from the current master branch, 25772c9e1d2d5239e1f7f83f6ea0c4d7d1f1df4a

K, I will try to allocate some time for review.
Sequencer is not worked on as much as other modules, so there is little chance of conflicts with your patches.

@Richard Antalik (ISS) Reading through; I'm seeing a lot of conflicting communication here.
A design task clearly detailing the goals, timeframes, scopes and functionality would be desirable, so other developers and uses can see what you have planned and what you intend to fix and work on. (Something like T46049: Assets Integration in Blender with T59540#583476)

In D3597#96377, @Christopher_Anderssarian wrote:

A design task clearly detailing the goals, timeframes, scopes and functionality would be desirable

Got this.
I intended to create some roadmap, I can define goals and abstract functionality.
Currently I should be working on prefetching, with aim to get it up and running in 2.8 release, which should be doable as code is already working.
I delved into a lot of stuff lately, maybe unnecessarily, well...

Ok so I will try to do some writing probably next week when I will have nothing better to do in work. I will probably state "obvious" things, which are broken, with solution being "it has to be dealt with". So I won't be able to provide precise timeframe, but it will be roadmap nonetheless...

In D3597#96377, @Christopher_Anderssarian wrote:

A design task clearly detailing the goals, timeframes, scopes and functionality would be desirable, so other developers and uses can see what you have planned and what you intend to fix and work on. (Something like T46049: Assets Integration in Blender with T59540#583476)

Done : T60912

For now I'm trying to focus on small changes that improve the user experience with maximum chances of being ready for the 2.8 release