Change Details

This is a draft, any idea or suggestion is welcome. To simplify VSE code and make new feature-friendly framework I would propose a following set of changes: Once Troy Sobotka suggested implementing OTIO. If it does 90% of what this should do, it may be worth trying to implement. Although even custom API should be doable under 1-2K LOC. ##Metadata / timeline separation Each sequence struct contains metadata mixed with timeline data. Metadata would be for example video stream length, framerate, format, ... Timeline data would be position on timeline, length on timeline, framerate on timeline, ... - On file load metadata are read, and timeline data are calculated once, by _arbitrary_ algorithm. - Any change in metadata(replacing file) must not result in any change of timeline data immediately. - Timeline data may have it's own struct, that will be consistent(as much as possible) across all sequence types. ##Timeline API - This API should provide atomic functions for moving strip around the timeline. These are the only functions that may be/contain sequence type specific code. - Other functions built on top of these atomic functions can also be provided. - Functions should be as universal as possible. - Developer should be able to manipulate with strip on higher level, without knowing anything about strip. -- (Move strip by offset, move with handles, add n still frames...) - Developer should also be able access timeline data directly, for features, that aren't covered by API functions. - In some cases it may be beneficial to use other units, than number of frames. It would be nice to create unitless code, but this is really optional. - Partial frames and custom playback speeds must be supported. This is the main reason why this change is needed. Support in sequencer itself is separate task. - To support custom playback rates, when recalculating offsets, sequence metadata are needed. Metadata must never be changed, unless user changes file or overrides some property. Metadata must always be referenced and never assumed / stored as timeline data. ##Partial frames Multiple sequences can have different frame/sample rates. In order to be able to do precise cuts, we need to support partial frames. For sequencer this means to be able to set playhead in between 2 frame boundaries, and displaying this position sensibly. Visually and numerically. For timeline API this would mean to be able to store positions and offsets with greater resolution, then its framerate(there may be beneficial to use different units) and report, where next(nearest) partial frame is to sequencer, when partial frame position is requested. Open question is, if we allow strips to start at partial frames, or only end at partial frames. Personally I would not allow to start at partial. Next open question would be maximum resolution. It may be beneficial to allow resolution up to sample rate of sound strip, but only for sound editing. This would allow us to edit "pops". Or maybe to cut sound, when "crossing zero" for preventing "pops", but in this case it may be better to create operator to add quick fade ins and fade outs. In audio editing it is best to do cut at "zero cross" and add 20-100 sample long fade. I would recommend to implement option for this quick fade for @nexyon if not implemented yet. Also question for @nexyon: can you support such precise cuts, If we define exact behavior of timeline? ##Playback speed animation This is more a TODO for very adventurous developer :) In sequencer, visualization of such events should be "pre-animated" eg. when animating sound pitch, rather then show changes in waveform in real time, waveform should be drawn pre-stretched locally to correspond playback. Speed effect currently doesn't support animation of playback speed, but it should. this should be shown in timeline by drawing curve on strip, or by "frame markers" spreading apart/together or both. For timeline API this means basically the same - calculation of correct length and position of individual frames. Both cases will require "simulated run" of F-curves. This would correspond to doing partial integral arithmetically, if not in F-curves API, may be added.

This is a draft, any idea or suggestion is welcome. To simplify VSE code and make new feature-friendly framework I would propose a following set of changes: Once Troy Sobotka suggested implementing OTIO. If it does 90% of what this should do, it may be worth trying to implement. Although even custom API should be doable under 1-2K LOC. ##Metadata / timeline separation Each sequence struct contains metadata mixed with timeline data. Metadata would be for example video stream length, framerate, format, ... Timeline data would be position on timeline, length on timeline, framerate on timeline, ... - On file load metadata are read, and timeline data are calculated once, by _arbitrary_ algorithm. - Any change in metadata(replacing file) must not result in any change of timeline data immediately. - Timeline data may have it's own struct, that will be consistent(as much as possible) across all sequence types. ##Timeline API - This API should provide atomic functions for moving strip around the timeline. These are the only functions that may be/contain sequence type specific code. - Other functions built on top of these atomic functions can also be provided. - Functions should be as universal as possible. - Developer should be able to manipulate with strip on higher level, without knowing anything about strip. -- (Move strip by offset, move with handles, add n still frames...) - Developer should also be able access timeline data directly, for features, that aren't covered by API functions. - In some cases it may be beneficial to use other units, than number of frames. It would be nice to create unitless code, but this is really optional. - Partial frames and custom playback speeds must be supported. This is the main reason why this change is needed. Support in sequencer itself is separate task. - To support custom playback rates, when recalculating offsets, sequence metadata are needed. Metadata must never be changed, unless user changes file or overrides some property. Metadata must always be referenced and never assumed / stored as timeline data. ##Partial frames Multiple sequences can have different frame/sample rates. In order to be able to do precise cuts, we need to support partial frames. For sequencer this means to be able to set playhead in between 2 frame boundaries, and displaying this position sensibly. Visually and numerically. For timeline API this would mean to be able to store positions and offsets with greater resolution, then its framerate(there may be beneficial to use different units) and report, where next(nearest) partial frame is to sequencer, when partial frame position is requested. Open question is, if we allow strips to start at partial frames, or only end at partial frames. Personally I would not allow to start at partial. Next open question would be maximum resolution. It may be beneficial to allow resolution up to sample rate of sound strip, but only for sound editing. This would allow us to edit "pops". Or maybe to cut sound, when "crossing zero" for preventing "pops", but in this case it may be better to create operator to add quick fade ins and fade outs. In audio editing it is best to do cut at "zero cross" and add 20-100 sample long fade. I would recommend to implement option for this quick fade for @nexyon if not implemented yet. Also question for @nexyon: can you support such precise cuts, If we define exact behavior of timeline? ##Playback speed animation This is more a TODO for very adventurous developer :) In sequencer, visualization of such events should be "pre-animated" eg. when animating sound pitch, rather then show changes in waveform in real time, waveform should be drawn pre-stretched locally to correspond playback. Speed effect currently doesn't support animation of playback speed, but it should. this should be shown in timeline by drawing curve on strip, or by "frame markers" spreading apart/together or both. For timeline API this means basically the same - calculation of correct length and position of individual frames. Both cases will require "simulated run" of F-curves. This would correspond to doing partial integral arithmetically, if not in F-curves API, may be added. ##Links to relevant tasks / discussions: {T44808} {T53615}

This is a draft, any idea or suggestion is welcome. To simplify VSE code and make new feature-friendly framework I would propose a following set of changes: Once Troy Sobotka suggested implementing OTIO. If it does 90% of what this should do, it may be worth trying to implement. Although even custom API should be doable under 1-2K LOC. ##Metadata / timeline separation Each sequence struct contains metadata mixed with timeline data. Metadata would be for example video stream length, framerate, format, ... Timeline data would be position on timeline, length on timeline, framerate on timeline, ... - On file load metadata are read, and timeline data are calculated once, by _arbitrary_ algorithm. - Any change in metadata(replacing file) must not result in any change of timeline data immediately. - Timeline data may have it's own struct, that will be consistent(as much as possible) across all sequence types. ##Timeline API - This API should provide atomic functions for moving strip around the timeline. These are the only functions that may be/contain sequence type specific code. - Other functions built on top of these atomic functions can also be provided. - Functions should be as universal as possible. - Developer should be able to manipulate with strip on higher level, without knowing anything about strip. -- (Move strip by offset, move with handles, add n still frames...) - Developer should also be able access timeline data directly, for features, that aren't covered by API functions. - In some cases it may be beneficial to use other units, than number of frames. It would be nice to create unitless code, but this is really optional. - Partial frames and custom playback speeds must be supported. This is the main reason why this change is needed. Support in sequencer itself is separate task. - To support custom playback rates, when recalculating offsets, sequence metadata are needed. Metadata must never be changed, unless user changes file or overrides some property. Metadata must always be referenced and never assumed / stored as timeline data. ##Partial frames Multiple sequences can have different frame/sample rates. In order to be able to do precise cuts, we need to support partial frames. For sequencer this means to be able to set playhead in between 2 frame boundaries, and displaying this position sensibly. Visually and numerically. For timeline API this would mean to be able to store positions and offsets with greater resolution, then its framerate(there may be beneficial to use different units) and report, where next(nearest) partial frame is to sequencer, when partial frame position is requested. Open question is, if we allow strips to start at partial frames, or only end at partial frames. Personally I would not allow to start at partial. Next open question would be maximum resolution. It may be beneficial to allow resolution up to sample rate of sound strip, but only for sound editing. This would allow us to edit "pops". Or maybe to cut sound, when "crossing zero" for preventing "pops", but in this case it may be better to create operator to add quick fade ins and fade outs. In audio editing it is best to do cut at "zero cross" and add 20-100 sample long fade. I would recommend to implement option for this quick fade for @nexyon if not implemented yet. Also question for @nexyon: can you support such precise cuts, If we define exact behavior of timeline? ##Playback speed animation This is more a TODO for very adventurous developer :) In sequencer, visualization of such events should be "pre-animated" eg. when animating sound pitch, rather then show changes in waveform in real time, waveform should be drawn pre-stretched locally to correspond playback. Speed effect currently doesn't support animation of playback speed, but it should. this should be shown in timeline by drawing curve on strip, or by "frame markers" spreading apart/together or both. For timeline API this means basically the same - calculation of correct length and position of individual frames. Both cases will require "simulated run" of F-curves. This would correspond to doing partial integral arithmetically, if not in F-curves API, may be added. ##Links to relevant tasks / discussions: {T44808} {T53615}