Some more notes after testing the latest version of the patch, and from recent module meeting discussions:

• Further performance improvements are needed. It was mentioned that these are depending on D15496 to be committed.

• The inverted cavity auto-masking needs to be added in an abstracted way to the pie menu

The behaviour we discussed is that the "Cavity" and "Inverted Cavity" pie menu toggles are mutually exclusive.
Toggling Cavity will enable the cavity setting and disable the Invert sub-setting in the options.
Toggling Inverted Cavity will enabled the Cavity setting and enable the Invert sub-setting in the options.

This behaviour is unusual for pie menus in Blender but would lead to the most intuitive workflow.

• The "Dirty Mask" operator in the header menu and "Bake Automask" panel in the options need to be replaced with a "Mask from Cavity" operator in the header menu.

This operator should make use of the Adjust Last Operation panel and expose the same options as the auto-mask.
By default a toggle at the top should be checked to "Use Auto-Mask settings"

• The Invert setting of the cavity auto-mask should also invert the effect of the "Use Curve" setting.

This way the inverted cavity auto-mask will be correctly inverted. This should be more convenient and avoid unnecessary manual inverting of the curve.

• Holding Ctrl while dragging the Factor slider to snap to increments doesn't work. It will always snap to 0 or 1.

@Joseph Eagar (joeedh) I tried out the updated patch:

• Update the inverted curve behavior

My note on inverting the curve was probably misunderstood. The way it is now in the patch, it will apparently flip the curve horizontally, which just causes the Inverted cavity to be inverted back to normal.
That is not what I meant. I'm not sure what the inversion is called that I mean but it's essentially the effect of changing all curve point locations to 1-x and 1-y.
Both the normal and inverted cavity mask added together should result in a fully masked surface.

• Remove Bake Auto-Mask

I think we should also not use a "Bake Auto-mask" operator and instead make a focused "Mask from Cavity" operator. This will keep the settings focused and easy to understand.
There is no clear use in baking the auto mask for face sets, topology or boundaries, since this can be done much faster and more reliably with Expand.
So instead of a whole Bake Auto-mask dropdown in the options, we can add a "Convert to Mask" button at the bottom of the Cavity auto-mask options.
This would then execute the Mask From Cavity operator with the auto-mask settings.

• Add Curve UI to Redo Panel

In the redo panel for Mask from Cavity there is also no Curve UI yet. This should be possible to add right? The Bevel operator also has a Curve exposed int he redo panel.

• Rename "Use Default Settings"

I would also suggest to rename the "Use Default Settings" to "Use Auto-Mask Settings". This is just clearer since "default" suggests that it will switch to a Blender default values.
Instead it actually switches to the user defined settings in the sculpt mode options, as the tooltip states.

An open question I have is: If the redo settings are changed while the "Use Auto-Mask Settings" toggle is enabled, can the auto-mask settings be updated based on those changes?

Great! The updates work good! Only a few more notes:

• The Blur steps in the redo panel seem to only start at a value of 2. The value of 0 and 1 has the same effect.

• The Curve UI in the redo panel has a bugged element. It shows and enabled "X" toggle in the top left corner instead of the zooming icons.

• The "Mask From Cavity" operator in the header is always defaulting to not "Use Auto-Mask Settings". That's fine for the first time it's used. But if the user enables the toggle to use the auto-mask settings, the operator needs to remember that settings for future operations.

EDIT: Ths note was not solved like suggested but instead all settings are hidden when "Use Auto-Mask Settings" is enabled. This solves the issue at least somewhat.

• The "Bake Cavity Mask" in the options ignores the Invert and Use Curve settings. This needs to be fixed.

• Since the "Bake Cavity Mask" UI is only relevant if the cavity auto-masking is enabled, we should make that context clearer.

I'd suggest moving the dropdown up so it is directly below the auto-masking section. We should also hide it when no cavity auto-masking is used.

• The operator for creating the cavity mask should also ideally be renamed to "Mask From Cavity", or something along those lines, like the menu operator, instead of "Bake Cavity"

It's only really being "baked", when put in context with the auto-masking settings. Baking is also typically associated with the Baking section in the render properties.
It's fine to keep the naming of "Bake" in the sculpt mode options UI and button right under the auto-masking settings IMO. This note is just about the name of the operator in general and the title in the redo panel.

Some more issues that I've found, some of them caused by recent updates:

• The naming of toggles in the Adjust Last Operation panel changes depending on the "Use Auto-masking Settings" toggle.

These should stay consistent. I'd suggest to use the naming from when the "Use Automask Settings" is toggled off.

• Toggling "Invert" in the Mask From Cavity operator options, toggles the Cavity (Invert) auto-masking mode, if the toggle "Use Automask Settings" is enabled

We should avoid this unintentionally toggling auto-masking modes. This should only affect the settings of the auto-masks. Not if they are used.
So I'd suggest that the "Invert" toggle in the Mask From Cavity operator does not affect any setting in the auto-masking options, even if "Use Automask Settings" is enabled.

• The cavity auto-masking settings in the brush settings need to be completely independent from the settings in the options panel.

This means that settings like Cavity Factor, Cavity Blur and the Use Curve toggle and the Cavity Curve must not be synced.
The brush settings needs to be able to set their own properties, to make the brush behavior predictable.

• We need to rethink the Mask From Cavity panel in the options.

This part of the UI is supposed to always use the auto-mask settings. Like mentioned in a previous note, this should be made clearer.
A big issue right now is also that it duplicates the same UI even further in the properties.

Here's my suggestion:

Instead of displaying the entire options from the Mask From Cavity operator as a sub-panel, I suggest to only expose the operator as a button, directly under the cavity auto-masking options.
It's also important that:

• The Mask From Cavity button is only displayed if either Cavity or Cavity (Inverted) is enabled
• The operator, when used in the options panel, will always default to have "Use Auto-Masking settings" enabled
• If the user wants to change the settings of the operation, like mix Mode or Mix Factor, they can do so in the Adjust Last Operation popup.

Some last issues I've found.

• Pie Menu arrangment is broken (Now there are two Topology options and no Inverted Cavity)

I modified the UI a bit to make it clearer when a brush automasking setting is overriding a scene one.

• Let's leave this out of this patch. This makes the UI very messy at the moment and iterating on this would just block the patch from landing in master further.

This includes greying out toggles and UI areas and changing the toggle names in the properties and pie menus. We can just handle all of this as a separate patch.

• A small note is just that the button "Mask From Cavity" needs a bit more spacing below it.

Also please adress the inline comments about the tooltips.