[Boost-users] [Multi-index] hash_unique memory consumption vs std::map

Subject: [Boost-users] [Multi-index] hash_unique memory consumption vs std::map
From: Michael Fawcett (michael.fawcett_at_[hidden])
Date: 2008-11-04 18:03:09

In my application I need to load millions of vertices into a graph.
I'm using BGL for this.  Each edge has around 20-30 attributes
associated with it and each attribute has a unique ID (not necessarily
sequential or 0-based).

I'm currently using this multi_index_container for the attributes:

typedef detail::mutable_pair<long, _variant_t> id_attribute_pair_type;
typedef boost::multi_index_container
<
        id_attribute_pair_type,
        boost::multi_index::indexed_by
        <		boost::multi_index::hashed_unique<boost::multi_index::member<id_attribute_pair_type,
long, &id_attribute_pair_type::first> >
	>
> map_type;

which emulates:

typedef std::map<long, _variant_t> map_type;

If I check memory consumption after loading the graph using
multi_index I'm at 190MB.  Using std::map it's at 124MB.  I realize
this is because std::map is perfectly sized and a hashmap will grow in
chunks, but is there a way I can reserve a size ahead of time since I
know exactly how many attributes there will be?

My graph loading performance does not seem to change between
multi_index and map, however subsequent search operations
(shortest_path, etc) seem to favor multi_index.

Thanks for such a great library.  I've used it in many projects.

--Michael Fawcett