Page MenuHome

VSE UX: F-curve drawing on Speed effect strips
AbandonedPublic

Authored by Peter Fog (tintwotin) on Oct 21 2019, 1:45 PM.
Tokens
"Love" token, awarded by wilBr."Love" token, awarded by kursadk."Love" token, awarded by HEYPictures."Love" token, awarded by pablovazquez."Love" token, awarded by AndyCuccaro."Love" token, awarded by dulrich."Love" token, awarded by mon."Love" token, awarded by amonpaike."Like" token, awarded by davidmcsween.

Details

Summary

Drawing Changes:

  • F-curve drawing for Stretch, Multiply, Length and Frame Number.
  • Value drawing when no keyframes for Stretch, Length and Frame Numbers.
  • Slightly less transparent f-curve color to keep it visible over ex. thumbnails.
  • Draw line at zero in the theme grid color.

General view of the new drawing for each speed effect mode:

Detail of the horizontal zero (blue) line(later changed to black grid color):

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
  • Rebase master
  • Cleanup: Clang-format / Remove trailing space / Deduplicate code
  • Versioning: 2.93.18 and Fixes

Sorry for the delay.
Since I'm not familiar with this code or the feature, I'm not sure where to start looking.
And I fear that this effect has not been maintained for a long time (so I don't know if there is an active developer who is familiar with this area.)
In my case I would need to first:

  • Study the original feature;
  • Study the original code;
  • Analyze the new code;
  • Create a test file;
  • Test the patch.

This takes a lot of time.
It would be nice if you could create a test file so we can test this feature.

As mentioned above, I've abandoned this patch, I'm out of time and motivation, so someone else will have to deal with incredible weak UI/UX.

  • Fix versioning: The enum is not bitwise / Remove unused flags
  • Cleanup: Use ELEM macro
  • Cleanup: Rename New 'Sequence' membrers (speed_frame_number_fader -> speed_fader_frame_number and speed_length_fader -> speed_fader_length)
  • Cleanup: Use ELEM macro
  • Fix versioning and updates

Nice to have(but currently not implemented): Auto adjusting of endframe, when using Multiply or Boost.
(A previous attempt: https://developer.blender.org/D4894)

Peter Fog (tintwotin) resigned from this revision.Jun 4 2021, 3:32 PM
  • comment accidentally removed

In Frame Number or Length modes the f-curve areas drawn outside of the frame are "hold"(freeze frame) areas. Since "hold" is illustrated with a darker color, these areas could be drawn like this:

As for the Stretch mode, basically two durations are needed: One for the source and one for the target. As it is now, the source and the target in and out points are confusing mess, ignoring trimmed in and out points of the source strip.

A way to solve this is to use the the trimmed in and out points of the source strip as source duration and use the Speed strip as target duration, keeping the same frame for start and end of both strips. This means that the speed strip should have handles and it's durations should be adjusted independently of the source strip, however it should be anchored/parented to the source strip, so they will be moved together around in the sequencer.

Richard Antalik (ISS) requested changes to this revision.Jul 8 2021, 3:56 AM

Hi, I have talked with @Hjalti Hjálmarsson (hjalti), we both agree, that these changes in functionality look reasonable.

This patch must be split into 2 parts - fcurve drawing and changes in current functionality. Also noticed, that files DNA_ipo_types.h and ipo.c are appended. These are legacy systems that should be only used for versioning for old files. If these changes are necessary, it may be good idea to do some cleanup prior to this change.

This revision now requires changes to proceed.Jul 8 2021, 3:56 AM

In fact the changes to ipo are not necessary, as it is only used for versioning.
Since it's more appropriate to split into 2 patches, I created D11856 which only contains the functionality changes.
This one can be updated later with only the fcurve drawing changes.

Peter Fog (tintwotin) added a comment.EditedJul 9 2021, 5:38 PM

@Germano Cavalcante (mano-wii) @Richard Antalik (ISS) Great to see this is finally moving forward.

As mentioned earlier part of the confusion is due because it is not clear what in/out/durations are used to what calculations.
Imo, it would be more clear if the input in/out/duration was used from the movie strip and the target output in/out/duration was manually set/auto-set in the speed strip - as mentioned here: https://developer.blender.org/D6110#304895

This way the user always will know the final duration, because it's the duration of the speed strip.

What do you think about this idea?

  • Rebase on master
  • Many tweaks to f-curve drawing code
This revision was not accepted when it landed; it landed in state Needs Review.Jul 23 2021, 11:56 PM
This revision was automatically updated to reflect the committed changes.

I'm not sure what Many tweaks to f-curve drawing code means, but I guess it is code optimizations, because the functionality seems to be the same?

  • For improved UX, if the suggested auto/manual duration change of the Speed strip isn't thought of as a good idea, then please consider to draw the hold/freeze frame areas on the speed strip ex. in Multiply, when it is returning frame numbers higher than the source strip duration or by multiplying by zero, in Length and Frame Number, when higher or lower than strip range. That would be very helpful info for users.
  • A nice to have, is drawing of the keys on strip(the diamond), so users will know if they're working on the right key, when using ex. prev/next key.


Unrelated to this patch, but related to Speed strips, there are still problems with the preview isn't updated when adding a new key. Could inserting/changing a key force a ctrl+r refresh, instead of having to manually do it?

I'm not sure if you have done something about the Multiply calculations, but now it is working as expected, were positive values will play forward, zero will freeze and negative will play backwards(@Richard Antalik (ISS)):

I have already applied this patch so it's probably best to respond (already...).

My version doesn't work for some reason(I don't see curves), but I should address concept first anyway:
I don't think, that drawing line in strip with static speed value is good idea (like in stretch mode). Perhaps this should be written as text, which would be more readable? This way user can take this value as baseline and switch to miltiply mode to create more complex setup.
Property ranges here are not limited to 0-1 range and so it is only practical to draw normalized curves. I assume there is API for this, and that it is fairly fast.

I probably wouldn't find this feature very useful in terms of visualizing what will be done just from looking at curve, but perhaps it's useful in terms of quicker orientation and showing complexity of animation.

@Richard Antalik (ISS) For stretch the default speed could be a centered blue line indicating the normal speed. Like this:


(Snapping at hold is quite important, because that would mean snapping to normal speed)

BLI_assert(v->speed_control_type == SEQ_SPEED_STRETCH);
const int target_strip_length = seq_effect_speed_get_strip_content_length(seq->seq1);
if ((seq->seq1->enddisp != seq->seq1->start) && (target_strip_length != 0)) {
  curve_val = (float)target_strip_length /
              (float)(seq->seq1->enddisp - seq->seq1->start) / 2.0f;
  ymin = -1.0f;
}

For testing the curve drawing:
I'm using this clip: https://www.pexels.com/video/aerial-shot-over-a-white-sand-beach-4135116/
In this .blend file:

I'm not sure what Many tweaks to f-curve drawing code means, but I guess it is code optimizations, because the functionality seems to be the same?

Well noticed. Those accidental commits messed everything up.
Corrected now.

I'm not sure what Many tweaks to f-curve drawing code means, but I guess it is code optimizations, because the functionality seems to be the same?

Well noticed. Those accidental commits messed everything up.
Corrected now.

I still don't see any functional changes? Are there any?

How do you like the Stretch drawing suggestion? As shown, it is very easy to implement.

I still don't see any functional changes? Are there any?

There were no functional changes.

How do you like the Stretch drawing suggestion? As shown, it is very easy to implement.

Sounds good, but I have to look deeper into the current solution.
(I only updated the existing patch, I still have to analyze all the cases).

I'm sorry. Rechecking your patch, I don't get any f-curve drawing at all.

I'm sorry. Rechecking your patch, I don't get any f-curve drawing at all.

I can't replicate this problem.
The drawing appears as in the description here.

I'll check again when I'm at my computer again. Could a test build be made of this, so more users could test it?

[EDIT: It is working as advertised here, I don't know what caused the fallout]

Sorry, I must have been doing something wrong, I have re-checked and I see curves now. I see they are normalized too. I would probably suggest using different color for 0 line. I find it distracting when it is same as current frame. Personally I would go for 1px thin darker line in same hue as strip body. Or not draw it at all if it is still distracting.

@Richard Antalik (ISS) The horizontal line is drawn in consistency with the horizontal line in the graph editor(for quick check see the gif above).

@Richard Antalik (ISS) The horizontal line is drawn in consistency with the horizontal line in the graph editor(for quick check see the gif above).

That line in graph editor is not line for 0 value, it is cursor position line, similar to one for time. You can move it with shift+right click.

I would probably suggest using different color for 0 line. I find it distracting when it is same as current frame. Personally I would go for 1px thin darker line in same hue as strip body. Or not draw it at all if it is still distracting.

Nb. 1px lines are supposedly invisible on Macs.
And color strips are already using a darker line to
separate the text area and the content area in
strips(a concept which could be used in all types
of strips including ex. Movie strips with thumbnails
drawn):


Not drawing the line at zero will make it impossible
to see when ex. the strip is playing the frame order
in reverse, so imo it is pretty crucial to make it visible.
Maybe the color of the x-axis in the 3D View
can be used?

I would probably suggest using different color for 0 line. I find it distracting when it is same as current frame. Personally I would go for 1px thin darker line in same hue as strip body. Or not draw it at all if it is still distracting.

Nb. 1px lines are supposedly invisible on Macs.

That can't be correct.

Yes, something like this is what I would suggest. It doesn't distract, but you see it quite well.

I think this would be also quite distracting because of contrasting colors. Still better than color of playhead though.

Yes, something like this is what I would suggest. It doesn't distract, but you see it quite well.

I hope you read the text too? This line is already existing. If it is implemented on all strips, you'll basically have two lines on strips looking like that if f-curves are drawing - unless the zero line isn't visible, and users may think that the separator line is the zero line.

Germano Cavalcante (mano-wii) planned changes to this revision.Aug 16 2021, 10:00 PM

It's always good to improve the design, but I'm not sure what changes have been decided here.
I'm glad we made progress on the speed effect interface, but I confess that I'm not paying much attention to this part of the original patch.

So, if anyone is interested in moving things forward, feel free to command and edit the patch ;)

Peter Fog (tintwotin) commandeered this revision.EditedAug 25 2021, 3:16 PM
Peter Fog (tintwotin) updated this revision to Diff 41082.
  • Changed color of zero line to grid color.
  • Added zero line for Stretch.
  • Made the f-curve drawing slightly darker to keep it visible on the upcoming thumbnails.

Peter Fog (tintwotin) retitled this revision from VSE UX: Make Speed Effect strips more user friendly. to VSE UX: F-curve drawing on Speed effect strips.Aug 25 2021, 3:24 PM
Peter Fog (tintwotin) edited the summary of this revision. (Show Details)Aug 25 2021, 3:29 PM

LGTM.
One thing I worry a little about is whether the complexity of the drawing can affect the VSE FPS (since we need to calculate and read f-curves).
Adding a cache for the same batch to be read if there are no changes would be ideal.

But it is almost certain that the penalty in performance is negligible. And this is the same drawing solution seen for other strip types.

LazyDodo did a build with this patch for you guys and users to test. It's D6110 and can be downloaded here: https://builder.blender.org/download/patch/
A test blend file can be downloaded here:


(Switch between the different speed control modes, after selecting the speed strip, to see the various f-curve drawings)
Unrelated to this patch is there something broken with Multiply mode, where the last frame gets "stuck": https://developer.blender.org/T90920

  • Changed color of zero line to grid color.

Looks better, but even better would be if it used color of strip body with lower intensity, so with custom themes this won't look out-of place.

  • Added zero line for Stretch.

I still think it would be better to just print speed factor exactly. I would only draw f-curves really.

It's always good to improve the design, but I'm not sure what changes have been decided here.

Above are my only issues as far as usability goes. That said, I would appreciate third opinion on these.

source/blender/editors/space_sequencer/sequencer_draw.c
95

You shouldn't include this file, only SEQ_xxx.h should be included in other modules. seq_effect_speed_get_strip_content_length can be moved to SEQ_time.h

  • Changed color of zero line to grid color.

Looks better, but even better would be if it used color of strip body with lower intensity, so with custom themes this won't look out-of place.

Maybe you could make a mock-up of how visible you want it?

  • Added zero line for Stretch.

I still think it would be better to just print speed factor exactly. I would only draw f-curves really.

Can you elaborate on why you want to draw the speed factor on the strip, if you want it exposed then is should be in the sidebar?
In Stretch mode the "zero" line is an indication of playing with normal speed, imo, that's quite useful, since it makes it very understandable if you're getting faster or slower video after the stretch.

I don't understand the include comment, and I don't think it is something I've touched, maybe @Germano Cavalcante (mano-wii) can help out with this?

Can you elaborate on why you want to draw the speed factor on the strip, if you want it exposed then is should be in the sidebar?

Sure, let's say you have sped up strip where you used stretch mode. Now you want composite another strip, but you must syncronize playback rate.


Now which one of these visualizations does show you more information?

To me this looks that this is really visualization for sake of having visualization. In addition you may have strips that look like they are muted/hidden, while they are not.

I don't understand the include comment, and I don't think it is something I've touched, maybe @Germano Cavalcante (mano-wii) can help out with this?

Now seq_effect_speed_get_strip_content_length is intended to be only utility function for internal sequencer logic. Drawing should not directly use internal logic of any editor. If this function has to be exposed to outside world, I would perhaps change it a bit. Not sure if I would change only it's argument to be parent effect or perhaps try to keep math inside of sequencer and only provide value for drawing, because drawing code should not contain much of internal logic.

My point with inline mainly was that this is not allowed unless absolutely necessary in general.

source/blender/editors/space_sequencer/sequencer_draw.c
1380

I would use immUniformColor4ub(0, 0, 0, 128);

Regarding line color, it's not about how visible I want it to be, it's about it's color. This way you can change grid color or strip color, it will always look the same.

Peter Fog (tintwotin) updated this revision to Diff 41157.EditedAug 27 2021, 12:45 PM

Add: The requested line color.

When in stretch, and users add a Speed strip, the line is at normal speed. When users then change the duration of the source strip ex. shortening it, they will see that the curve is moving above the normal speed line, indicating that the strip will now play faster, same thing when they stretch out the duration, then the curve will go below the line indicating that the speed will be slower than normal speed. I think this is the main use of it - making it more intuitive what will happen.

It seems to me, that what you're after is something else, which will need to expose the "Target Frame" value at the current frame, but afaik this is not available to expose in the sidebar(where it belongs, imo). In addition to this, you'll need to be able to change this value and have it linked to source strip duration to be able to reuse values from one strip to another. An easier way to solve this would be to use the same value in Multiplier mode for both strips.

On the f-curves looking like hold areas, I already tried to address this in the vse chat with a proposition to change the visual identity of hold areas:

Previously uploaded the wrong diff.