We can't really propose using dragging and dropping for moving modifiers without having something like this in my opinion.
This adds a modal operator called Edge Pan, which is meant to run invisibly while something inside a region is being dragged. This patch applies this to dragging panels, but it can be used elsewhere too.
Here I showed it with a small region just for a better aspect ratio. It looks like it skips a few times in the video, that's just a low framerate.
The speed and delay are easily adjustible, and the speed also increases subtly the start value to a max to make it feel more interactive and less robotic.