The approach is fairly simple, just apply an edge detection filter to the view normal and scale the brightness based on that.
The overlay is disabled at object boundaries to avoid dark lines around objects.
Generally, this implementation follows the proposal of @Bartosz Moniewski (monio) at https://blender.community/c/rightclickselect/J9bbbc.
The changes are:
- Radius of two pixels instead of one - might not be better, though, needs some feedback
- Options to reduce the strength of both ridges and valleys
- Tweaked function for the strength reduction (the original method actually had a local maximum, resulting in a brighter line inside valleys)
- Multiplication for blending instead of overlay, which doesn't work reliably with scene-referred intensities
- Renamed to point out the distinction between it and the SSAO-based cavity overlay
