Similar to other changes to ID remapping, gives huge speedups in some
cases, like certain types of liboverride creation.

Case from T96092: Dynamic Overrides Use Case - Giant object-based 'rig' with a few key controlers. goes from 1725 seconds (almost 30 minutes) to 45
seconds to generate the liboverride, on my machine.