Page MenuHome

MultiSpan datastructure for iterating over several spans transparently
AbandonedPublic

Authored by Victor-Louis De Gusseme (victorlouis) on Apr 10 2021, 1:58 PM.

Details

Summary

Very WIP patch to address an issue I encountered when writing the Attribute Statistic node. This patch is basically a proof-of-concept and a starting point for other ideas and feedback.

Diff Detail

Repository
rB Blender
Branch
spanspan-iterator (branched from master)
Build Status
Buildable 13980
Build 13980: arc lint + arc unit

Event Timeline

I'm not entirely sure if this abstraction is worth it. Using it just as a forward iterator might be fine though, not sure. Maybe the concept can be generalized.
I wonder if adding a random access iterator here provides any real benefit, but it could be worth a try. It feels like it would add quite some overhead to element access when no expensive preprocessing is done.

One thing I don't understand is why this inherits from Span. A multiple span is not a span.

The overhead for each element access is not ideal, but when you need this functionality, the alternative is usually copying all the data in all the spans.

The random_access_iterator is required for std::nth_element() which I want to use for computing the median in the Attribute Statistic node.

My thinking for the inheritance from Span was that this way you could just write your functions to work with Span, but you can also call them with a MultiSpan. You'd only need a single function for both, and let the iterator handle the iteration. For the user, a MultiSpan should behave like a regular Span as much as possible.

Example of how the inheritance could allow prettier code:

int sum(Span<int> span) {
    int sum = 0;
    for (int value : span) {
        sum += value;
    }
}

Vector<int> a = {1, 2, 3};
MutableSpan<int> a_span = a;

Vector<int> b = {4, 5, 6, 7};
MutableSpan<int> b_span = b;

Vector<MutableSpan<int>> spans_vector = {a_span, b_span};
Span<MutableSpan<int>> spans = spans_vector.as_span();
MultiSpan<int> multispan = MultiSpan<int>(spans);

sum(a_span)
sum(multispan)

As Jacques pointed out, what I explained in my previous comment is not possible in C++.

The way to decouple algorithms from datastructures in C++ is to use templated functions that take iterators as arguments.

So what I'll try is to define only a special iterator to iterate over a Span<MutableSpan<T>> (so no MultiSpan class anymore). The statistics for the Attribute Statistic node will then be rewritten to take iterators as inputs.

  • Sum test that shows this is not how inheritance works in c++
  • removed sum() and calculate sum with std::accumulate() instead

Current plan: keep MultiSpan, it's job is to provide a .begin() and .end() for the iterator. Then use algorithms that work with iterators, see the sum test for an example with std::accumulate().

Closing this revision, and abandoning the idea of a multispan or of a custom iterator that loops over multiple spans.

I did some tests, and the performance cost of a more complex iterator is just not worth it. The more pragmatic solution is to just copy the data when needed. The overhead of copying is generally negligible compared to the cost of the operations that will happen on the data afterwards.