To have an idea of what's leaking you can call blender with -D --debug-memory. It will print string that is passed to MEM_CallocN that allocates memory for data.

I would suggest to follow my rant. I think it can bring you to the end faster.

This code can be broken down to 4 sections:

Controller
Use job system to create background task that will run builder

This may look a bit complicated at first, but it's very simple.
For start you play around by modifying seq_proxy_build_job() and running it manually. This function can be quite simplified.
Important steps are:

• create/get existing job with WM_jobs_get()
• set customdata with WM_jobs_customdata_set() - this can be simple struct with your desired range.
• setup timer with WM_jobs_timer() - will update progressbar and redraw timeline
• setup callbacks with WM_jobs_callbacks() - callback should be function in seqprefetch.c that will call your version of seq_prefetch_start(). For start this can be simple function that will do hello world to check that everything is running. Start callcack will run in new thread. end callback will do cleanup (free stuff).

Just a note: By design, each owner(usually editor) can run only 1 job at the same time. Also see wm_jobs.c

Builder
Setup render engine to render strip and store images in cache.

I would recommend to use seqprefetch.c file and it's functions. It's pretty much designed to do this kind of job.
To invoke it, just copy and modify function seq_prefetch_start() to work on lower resolution(context->rect) and remove threading stuff - that is done for you by job system.
You may want to use slightly different version of seq_prefetch_frames() that would render every n-th frame or work on list, but we can deal with that later. as a first step you can render just 1 frame by simply running

ImBuf *ibuf = BKE_sequencer_give_ibuf(&pfjob->context_cpy, cfra, 0);
BKE_sequencer_cache_free_temp_cache(pfjob->scene, pfjob->context.task_id, cfra);
IMB_freeImBuf(ibuf);

Viewer
To get frame stored in cache, just call:
ibuf = BKE_sequencer_cache_get(context, seq, cfra, SEQ_CACHE_STORE_COMPOSITE);
Make sure, that you are requesting same resolution and frame(s), that are built by builder (building and viewing context must match)
You can modify code you already have and manually set context to same size to see how and if this would work. You will have to manually enable caching of composite images.
You will also have to 'build context' - follow code in sequencer_draw :

SeqRenderData context = {0};
BKE_sequencer_new_render_data(bmain, depsgraph, scene, rectx, recty, proxy_size, false,&context);
context.view_id = BKE_scene_multiview_view_id_get(&scene->r, viewname);

Storage
This should be done automagically by D5524: VSE: Disk cache

TODO
We will have to do some hacks ehm implement flags to identify thumbnail thread (see eSeqTaskId - just add new one), and override (ignore) global SEQ_CACHE_STORE_COMPOSITE setting for thumbnails. Thumbnail count will be limited by cache size, there will probably be some conflict with "main" content, this can be easily fixed by allocating some space just for thumbnails and modifying math in BKE_sequencer_cache_is_full() to something like

if(thumb_job){
    return memory_total < cache->memory_used;
}
else {
    return memory_total < cache->memory_used + 16 * 1024; /* 16MB reserved for thumbnails */
}

Recycling will also have to be either modified, but that shouldn't be big problem. Pretty much just remove anything that's not in view.

Finally to close the loop - when thumbnail is requested and any is missing, just start the job. Thumbnails, that are already built will not be overwritten and quickly skipped. When done, free the job data and quit.

Overall
By doing this, you get quite robust system, that will affect UI in minimal extent (it will affect UI because sequencer isn't fully decoupled - UI is locked during rendering), asynchronous and mainly instant image displaying, standardized storage with automatic invalidation provided by cache.

I would implement these blocks starting by simple first tests, then simple proof of concepts, and add functionality from there.
I would recommend to start with viewer, then controller and finally builder.
Storage will be done later by me, no need to do anything special to make it work.

Not sure if @Guillaume M (mathers) still wants to work on this, you can both work on different part.

If you will need help, I will do what I can.

Thank you very much for the detailed explanation. I pinged @Guillaume M (mathers) on the forum in case he still wanted to work on this, to not steal his work.
With these explanations, the way forward is pretty clear. I'd start having just 1 thumbnail at the start of a strip as you said, to not have to prefetch/render too much footage, even if it's at a low resolution - as there's that 1 job limitation + possibly lock happening.

The first step is in place: requesting thumbnails from the full composite cache. The next step is to try and render low-res thumbnails using the job system, and then have some memory dedicated to thumbnails.

For now, I'm only drawing the first frame of each strip to avoid slowing down rendering during development.

I've started working on the controller. I'll just need a bit of time to add a working commit as I'm busy with work and upcoming conferences, but I started to code something based on seq_proxy_build_job()

Could you just tell me in which file I should put it? The function to draw thumbnails is in sequencer_draw, and for now, I placed my seq_movie_thumbnails_build_job function in the same file, right next to one another. Should it be somewhere else?

Will this also work for movieclips?

In D5908#152661, @Eitan Traurig (EitanSomething) wrote:

Will this also work for movieclips?

It should work for all types.
Scene strips may be problematic, because we can not render them from thread without locking UI basically.

The preview for movie strips in the sequencer isn't working for me.

In D5908#152880, @Eitan Traurig (EitanSomething) wrote:

The preview for movie strips in the sequencer isn't working for me.

Well this isn't finished by any stretch now. I wanted to work on this long time ago, but I didn't got to this.