Page MenuHome

FFmpeg: Improve accuracy of sequential decoding
AbandonedPublic

Authored by Richard Antalik (ISS) on Jan 25 2022, 3:55 PM.

Details

Summary

If VFR movies are sequentially decoded, presentation timestamp is not
taken into consideration. This means, that frames with varying duration
will be presented at constant intervals and eventually will fall out of
sync with time at which they should be presented.

During rendering, internal state is reset which causes decoder to seek
in stream, which does consider presentation timestamp. This means that
there can be mismatch between preview and rendered image.

Check if presentation timestamp matches requested timestamp. If not,
reset decoder state, seek to keyframe and scan for requested timestamp.
This does resolve mismatch between image in preview and rendered image.

Note: This patch is essentially same as D13583, but much less efficient.
It can cause strip length to be detected incorrectly because some frames
will be displayed for longer time. Number of frames in stream is
unknown. unles they are all decoded. this is why currently timecodes are
used instead.

Diff Detail

Repository
rB Blender
Branch
more-accurate-playback (branched from master)
Build Status
Buildable 20163
Build 20163: arc lint + arc unit

Event Timeline

Richard Antalik (ISS) requested review of this revision.Jan 25 2022, 3:55 PM
Richard Antalik (ISS) created this revision.

Here is sample file you can examine:

Use provided .blend file where VSE cache is disabled, otherwise it will mask these issues.

In master, you can see that video is out of sync with sound, if you let it play for a while, stop and seek 1 frame back, it will jump quite a bit. If you build 100% proxies, you should see different image. If you set timecodes to record run, strip will suddenly become twice as large. Only after you enable timecodes, you will experience what is correct behavior.

In this patch, most of incorrect behavior is still in place. what is resolved is mismatch between preview image and rendered image.

with D13583 movie will be in sync with sound, seeking will work well ,but strip length will be incorrect. Strip length is currently only problem to resolve. Otherwise it would be perfect patch to resolve all these issues. There is no reliable data to base calculations on as far as I can see. Because of that I would consider that quite risky change that may have to be reverted. This is reason why I am trying to resolve this issue with timecodes instead.

I have found solution to seek in VFR files now and will continue with D13583.