Re: [Boost-users] [Multi-index] hash_unique memory consumption vs std::map

Subject: Re: [Boost-users] [Multi-index] hash_unique memory consumption vs std::map
From: Michael Fawcett (michael.fawcett_at_[hidden])
Date: 2008-11-06 16:09:44

On Wed, Nov 5, 2008 at 4:18 PM, Joaquin M Lopez Munoz <joaquin_at_[hidden]> wrote:
>
> You're in a 32-bit platform, right? Then, the first sizeof()
> should ideally be 24+4=28, not 32. I think you're having a padding
> issue due to the presence of a long type in id_attribute_pair_type.
> Can you try changing the order in which long and _variant_t
> are declared within id_attribute_pair_type? This might avoid
> padding and get you a sizeof(*(it.get_node()))=28. If not, well,
> we're out of luck.

Yes, 32-bit platform.  We're out of luck - I tried both ways.  32
bytes for both.

> Not entirely correct. The right formula is
>
>  n_edges*(n_attributes*sizeof(node)+ bucket_count*4)
>
> which in our case is
>
>  106,096*(22*32 + 23*4) = 84,452,416
>
>> In the std::map case, this results in 22 nodes at 12 bytes overhead
>> per node.  106,096 * 22 * 36 == 84,028,032 bytes.
>
> Not consistent with the previous calculation and your reported
> saving of 1% in memory consumption :-/ Also, you might want to
> investigate whether the overhead for std::map is 12 bytes per
> element or 16 bytes per element --the latter is more likely.

Well, the numbers are different because I was reporting total program
memory usage for both implementations while the formulas are for just
that particular data structure.  The percentages, however, should have
been consistent.  Not sure why they weren't, but I've since modified
the program so much that it would be difficult to go back and test.

I investigated the node structure that my std::map implementation is
using.  It has an overhead of 3 pointers and 2 chars.  This is with
Visual Studio 2005 SP1.

--Michael Fawcett