Geometry Boost List: [geometry] 1.57 Storing indices as values to save space.

Subject: [geometry] 1.57 Storing indices as values to save space.
From: Tim Finer (tim.finer_at_[hidden])
Date: 2014-11-26 13:37:12

Hi Adam,

I'm trying yet another approach, heh.  This time with 1.57.

I'm going to see if I can't just store the spatial data in a memory 
mapped vector and indices into it in an in-memory rtree, as long as I 
relax my requirement for the size of the data a bit.

What I thought I'd try is storing the smallest sized unsigned integer 
index possible (given the data) and reference a shared memory that I 
control the layout of.  As a test, I set up an rtree with the value of 
std::uint32_t and passed in an IndexableGetter functor that returns an 
indexable double triplet when it receives the std::uint32_t value.  I 
used the packing constructor to minimize the memory footprint.  This all 
works, except, it looks like the rtree really stores a pair of an 
indexable point and a value and not just the values.

I did some before and after memory snapshots and determined the size of 
the memory used is the same regardless if I set the value to an index, 
or if I fill the rtree with the double triplets directly.  My 
expectation was that the functor would be used each time that the 
indexable is needed, because the documentation said "Important: The 
translation is done quite frequently inside the container - each time 
the rtree needs it".

Except, that isn't really the case.  What really happens is that the 
rtree caches a temporary vector of the entire size incoming data.

pack_create.hpp, around line 155:
std::vector<entry_type> entries;
entries.reserve(values_count);

Where entry_type is a std::pair<point_type, InIt>, where 
sizeof(point_type) is 24 and InIt is the incoming value iterator.

I also tried dynamically inserting nodes too, but that used even more 
memory, although I didn't spend too much time looking at why.

So my question is, what is the purpose of the IndexableGetter?  Is there 
a way to trade off speed for space such that the rtree uses an 
IndexableGetter all the way through?  Was this by design?

Best,
Tim