VSE: Warn when seeking is incorrect
AbandonedPublic
Actions

Authored by Richard Antalik (ISS) on Jan 24 2022, 8:42 PM.

Details

Reviewers

Sergey Sharybin (sergey)
Francesco Siddi (fsiddi)

Maniphest Tasks

T95093: VSE: Variable framerate videos cause proxies to mismatch

Summary

Current movie seeking algorithm does not handle VFR media well. Problems
are not always obvious and can lead to different image being shown in
preview or render. This can be corrected by using generated timecodes
when proxies are built.

When seeking error occurs, raise this as warning to user.

This is implemented on decoding side as a state which can be polled
by function IMB_anim_seek_failed_check(). Polling does reset internal
state.

In sequencer, when seeking fails, pointer to Sequence is stored in
GSet within EditingRuntime. After rendering preview, all seeking
problems are reported. Strip can produce only one report during session.

2 types of warnings are produced to clarify issue - if timecodes are not
built, it is suggested to build them. If timecodes are available, but
not used, it is suggested to use them.

ref T95093

Diff Detail

Repository

rB Blender

Branch

auto_timecode (branched from master)

Build Status

Buildable 20138
Build 20138: arc lint + arc unit

Event Timeline

Richard Antalik (ISS) requested review of this revision.Jan 24 2022, 8:42 PM

Richard Antalik (ISS) created this revision.

Harbormaster completed remote builds in B20137: Diff 47391.Jan 24 2022, 8:42 PM

This is patch in addition to D13905, not replacement.

Here in addition to warning, I could do some action automatically as well, but in chat I was informed, that drawing code should not do any changes to data and as mentioned in D13905 it could be quite disruptive. But possibility is there.

Forgot to release lock

Harbormaster completed remote builds in B20138: Diff 47392.Jan 24 2022, 8:52 PM

Richard Antalik (ISS) mentioned this in T95093: VSE: Variable framerate videos cause proxies to mismatch.Jan 24 2022, 9:03 PM

I don't really think this is greatest thing to do. On an intuitive level, if the system is smart enough to detect that seeking issue happened, why can't it be smart enough to correct that as well? Such warnings is not something I'd expect to see in a video editor.

In D13911#369656, @Sergey Sharybin (sergey) wrote:

I don't really think this is greatest thing to do. On an intuitive level, if the system is smart enough to detect that seeking issue happened, why can't it be smart enough to correct that as well? Such warnings is not something I'd expect to see in a video editor.

D13583 does eliminate need for warnings like this. but it needs a lot of testing.

Maybe I should have disclosed this in description, that current system is not actually smart enough - during sequential decoding it wouldn't detect seeking problem, because it never seeks. I was relying rather on fact, that some seeking usually does occur. Having said that, it could be possible to check timestamp even during sequential decoding and on error restart decoding from keyframe. This may improve accuracy to acceptable level, so I will investigate this solution as well.

I say acceptable level, because with VFR media currently seeking will overshoot(by 1 frame usually), but so does rendering process. Therefore render would at least match preview. With D13583 no overshoot can happen therefore it is more correct.

In D13911#369668, @Richard Antalik (ISS) wrote:

Having said that, it could be possible to check timestamp even during sequential decoding and on error restart decoding from keyframe. This may improve accuracy to acceptable level, so I will investigate this solution as well.

I have tried to do this, and it does resolve mismatch between what is in preview vs what is being rendered. It does not resolve mismatch between proxy and non-proxy preview when no timecodes are used, but that should be resolved by D13905. The reason for this mismatch is that there are more frames in movie than duration * FPS.

You can look at this approach in D13917 and decide which would work better.