Boost logo

Boost Users :

Subject: [Boost-users] [Multi-index] hash_unique memory consumption vs std::map
From: Michael Fawcett (michael.fawcett_at_[hidden])
Date: 2008-11-04 18:03:09


In my application I need to load millions of vertices into a graph.
I'm using BGL for this. Each edge has around 20-30 attributes
associated with it and each attribute has a unique ID (not necessarily
sequential or 0-based).

I'm currently using this multi_index_container for the attributes:

typedef detail::mutable_pair<long, _variant_t> id_attribute_pair_type;
typedef boost::multi_index_container
<
        id_attribute_pair_type,
        boost::multi_index::indexed_by
        < boost::multi_index::hashed_unique<boost::multi_index::member<id_attribute_pair_type,
long, &id_attribute_pair_type::first> >
>
> map_type;

which emulates:

typedef std::map<long, _variant_t> map_type;

If I check memory consumption after loading the graph using
multi_index I'm at 190MB. Using std::map it's at 124MB. I realize
this is because std::map is perfectly sized and a hashmap will grow in
chunks, but is there a way I can reserve a size ahead of time since I
know exactly how many attributes there will be?

My graph loading performance does not seem to change between
multi_index and map, however subsequent search operations
(shortest_path, etc) seem to favor multi_index.

Thanks for such a great library. I've used it in many projects.

--Michael Fawcett


Boost-users list run by williamkempf at hotmail.com, kalb at libertysoft.com, bjorn.karlsson at readsoft.com, gregod at cs.rpi.edu, wekempf at cox.net