Instead of two functions, I'd be tempted to only have a single one (keeping the original name), with the FCurve arg as an optional last arg (e.g. FCurve *owner_fcu)

Ugh... since we're doing these checks in several places, I'd consider defining a macro for these checks within this function, and using it for these two lines and also on 959.

e.g.

#define BEZT_HAS_AUTOH(bz)  ELEM((bz)->h1, HD_AUTO, HD_AUTO_ANIM) && ELEM((bz)->h2, HD_AUTO, HD_AUTO_ANIM)

...

bool cycle = detect_cycle_modifier(fcu) && BEZT_HAS_AUTOH(first) && BEZT_HAS_AUTOH(last);

Looks like it's time to use braces here for the if/else, instead of keeping everything on single lines

See comments above (in the header). Is this function really necessary?!

lib_link() is only for stuff from another datablock. You should only need to fix the link in direct_link() since the F-Curve and F-Modifiers should be in the same datablock.

This should be handled in the remove_fmodifier function. Saves spraying this logic all over the code.

IIRC, a few of the other RNA callbacks may also benefit from having this info

This should probably be in add_fmodifier

This shouldn't be here (or in that other place in the UI code). It should instead be in remove_fmodifier)

It'd be better to have a rna_FModCycles_update(...) that looks something like

static void rna_FModCycles_update(...)
{
    if (fcm->curve) {
        calchandles_fcurve(fcm->curve);
    }
    rna_FModifier_update(bmain, scene, ptr);
}

Basically I treated the field as runtime-only, with any values coming from saved data being considered untrustworthy. It's simpler than special file format upgrade code.

I'm not really sure to perform handle recalculation in a low level function that simply deletes a modifier. It would be even more abstraction level hackiness. Note that calchandles_fcurve doesn't just set flags, it actually recalculates everything.

After more investigation, remove_fmodifier is also indirectly called when deleting an fcurve, after most of its data is already freed. Trying to recalculate handles from remove_fmodifier in this case will access deallocated memory.

Since remove_modifier can't update handles, neither should add_fmodifier.

The problem is that changing common properties like influence should also trigger an update, and using "rna_FModeCycles_update" for them would be strange.