[Boost-users] [BGL][Serialization] Reading a graph from a binary file: "vector subscript out of range error" and portability

Subject: [Boost-users] [BGL][Serialization] Reading a graph from a binary file: "vector subscript out of range error" and portability
From: Nicholas Mario Wardhana (mario.wardhana_at_[hidden])
Date: 2013-02-05 05:38:51

Hi all,

I am trying to read a graph from a binary file. This file was
previously generated by constructing a graph consisting of 1000
vertices and almost 10M edges. While vertices are properly loaded, I
encountered a "vector subscript out of range error" when edges are
being loaded in adj_list_serialize.hpp, line 97
(boost::tie(e,inserted) = add_edge(verts[u], verts[v], graph);). At
this line, u and v are all in the order of millions, whereas I only
have 1000 vertices.  When I ran the program without debugging, it
consumed the whole memory.

I put a not-so-minimal working code at the end of my e-mail. Could
somebody please point out any mistake in my code?

FYI, I use Boost 1.52, Visual C++ 2010, Windows 7 64-bit.

BTW, my aim is to generate save files which are fast to be read and
can be read on different computers. I am aware that while this binary
format can be processed fast, it is not portable. Can somebody please
tell me how non-portable it is? If there are only a few things to be
considered, for example architectural difference (32 vs 64-bit), it is
fine as I only need to generate two files for one graph. But, if there
are many of such factor, to the point that it is impossible to provide
numerous save files, is there any equally fast format which is more
portable?

Thank you!

Best regards,
Nicholas Mario Wardhana

//======================= CODE ===================================//

#include <boost/graph/adjacency_list.hpp>
#include <boost/graph/adj_list_serialize.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <fstream>

struct Point
{
    Point()
        : x(0.0f), y(0.0f), z(0.0f)
    {

    }
    float x;
    float y;
    float z;
};

struct Node
{
    Point point;
    float maxRadius;
    int motionType;
    int label;
    int ID;
};

struct Edge
{
    float cost;
    short int type;
    bool kept;
    bool skip;
    int nodeID1;
    int nodeID2;
};

namespace boost {
namespace serialization {

template<class Archive>
void serialize(Archive & ar, Point & v, const unsigned int version)
{
    ar & v.x;
    ar & v.y;
    ar & v.z;
}

template<class Archive>
void serialize(Archive & ar, Node & n, const unsigned int version)
{
    ar & n.point;
    ar & n.maxRadius;
    ar & n.motionType;
    ar & n.label;
    ar & n.ID;
}

template<class Archive>
void serialize(Archive & ar, Edge & e, const unsigned int version)
{
    ar & e.cost;
    ar & e.type;
    ar & e.kept;
    ar & e.skip;
    ar & e.nodeID1;
    ar & e.nodeID2;
}

} // namespace serialization
} // namespace boost

typedef boost::adjacency_list<boost::setS,
                              boost::vecS,
                              boost::undirectedS,
                              Node,                       // bundled data
                              Edge,                       // bundled data
                              boost::no_property,         // not needed
                              boost::listS> Graph;
typedef boost::graph_traits<Graph>::vertex_iterator NodeIterator;
typedef boost::graph_traits<Graph>::edge_iterator EdgeIterator;

int main()
{
    Graph g;

    for(int ID = 0; ID < 1000; ID++)
    {
        Node node;
        node.ID = ID;
        node.point.x = (float) ID;
        node.point.y = (float) ID;
        node.point.z = (float) ID;
        node.maxRadius = 1.0f;
        node.motionType = 0;
        node.label = 0;
        boost::add_vertex(node, g);
    }

    std::pair<NodeIterator, NodeIterator> vertexIterators = boost::vertices(g);
    NodeIterator nodeIterator1 = vertexIterators.first;
    for(; nodeIterator1 != vertexIterators.second; nodeIterator1++)
    {
        for(NodeIterator nodeIterator2 = nodeIterator1; nodeIterator2
!= vertexIterators.second; nodeIterator2++)
        {
            if(*nodeIterator1 == *nodeIterator2)
                continue;

            Edge edge;
            edge.cost = 1.0f;
            edge.kept = true;
            edge.skip = false;
            edge.type = 0;
            edge.nodeID1 = g[*nodeIterator1].ID;
            edge.nodeID2 = g[*nodeIterator2].ID;
            boost::add_edge(*nodeIterator1, *nodeIterator2, edge, g);
        }
    }

    std::cout << "Number of nodes = " <<  boost::num_vertices(g) << std::endl;
    std::cout << "Number of edges = " <<  boost::num_edges(g) << std::endl;

    std::cout << "SAVE!" << std::endl;

    std::ofstream filename1("binary_serialization.txt");
    boost::archive::binary_oarchive binary_oa(filename1);
    binary_oa & g;
    filename1.close();

    std::cout << "LOAD!" << std::endl;

    Graph g3;
    std::ifstream filename2("binary_serialization.txt", std::ios::binary);
    try
    {
        boost::archive::binary_iarchive binary_ia(filename2);
        binary_ia & g3; // error here
    }catch (std::exception& e)
    {
        std::cout << e.what() << std::endl;
    }
    std::cout << "Number of nodes = " <<  boost::num_vertices(g3) << std::endl;
    std::cout << "Number of edges = " <<  boost::num_edges(g3) << std::endl;

    int a = 0;
    std::cin >> a;
    return 0;
}