[Boost-users] [Graph] Distributed edge insertion

Subject: [Boost-users] [Graph] Distributed edge insertion
From: Sensei (senseiwa_at_[hidden])
Date: 2015-08-24 14:13:20

Dear all,

I am finding difficulties in understanding how distributed graphs work. 
My graph is simple, a string-valued property, with some edges, however, 
right now I have only one edge... if it works.

Since each process will add its own nodes, is it possible to avoid 
communication when adding nodes, given that each node owns its own 
vertices, and no overlap is possible between processes? This would save 
some communication avoiding re-distribution of nodes.

Can I add edges concurrently? Meaning, each process adds some edges. 
I've only seen examples with root adding edges, but that's limiting.

My last question is maybe easy. I'm adding an edge that connects two 
nodes, owned by different processes. Doing this results in a crash. It 
seems I cannot access properties with operator[] when traversing processes.

I'm posting the code below, along with the *crashing* output.

I hope you can help getting a grasp on PBGL!

Thanks!

#include <iostream>
#include <string>
#include <vector>
#include <cmath>
#include <list>
#include <functional>

#include <boost/config.hpp>
#include <boost/mpi.hpp>
#include <boost/graph/use_mpi.hpp>
#include <boost/graph/distributed/mpi_process_group.hpp>
#include <boost/graph/distributed/adjacency_list.hpp>
#include <boost/graph/iteration_macros.hpp>

using namespace boost;
using boost::graph::distributed::mpi_process_group;

// Useless class
class str
{
public:

     str(std::string s = std::string()) : s_(s)
     {
         // NOP
     }

     template<typename Archiver>
     void serialize(Archiver& ar, const unsigned int /*version*/)
     {
         ar & s_;
     }

     std::string s_;
};

// Allow indexing and constructing
namespace boost
{
     namespace graph
     {
         // Index based on the str class
         template<>
         struct internal_vertex_name<str>
         {
             typedef multi_index::member<str, std::string, &str::s_> type;
         };

         // Construct with str class
         template<>
         struct internal_vertex_constructor<str>
         {
             typedef vertex_from_name<str> type;
         };

     }
}

// Handy
typedef boost::adjacency_list<vecS, distributedS<mpi_process_group, vecS>,
                               bidirectionalS, str> dgraph;

typedef graph_traits<dgraph>::vertex_descriptor dvertex;

// START ME UP
int main(int argc, const char * argv[])
{
     boost::mpi::environment  env;
     boost::mpi::communicator comm;

     std::cout << "RANK " << comm.rank() << std::endl;

     comm.barrier();

     dgraph g;

     std::vector<dvertex> descriptors;

     // Each rank
     for(int i = 0; i < 10; i++)
     {
         str s(std::to_string(i + (comm.rank() + 1) * 100));

         std::cout << "rank " << comm.rank() << " adding " << s.s_ << 
std::endl;

         descriptors.push_back( add_vertex(s, g) );
     }

     BGL_FORALL_VERTICES(v, g, dgraph)
     {
         std::cout << "V @ " << comm.rank() << " " << g[v].s_ << " local 
" << v.local << " owner " << v.owner << std::endl;
     }

     std::cout << ">>> NUM VERTICES " << num_vertices(g) << " NUM EDGES 
" << num_edges(g) << std::endl;

     comm.barrier();

     if(comm.rank() >= 0)
     {
         int from, dest;

         bool letsCrash = true;

         if (letsCrash)
         {
             from = 1; dest = 0;
         }
         else
         {
             from = 0; dest = 0;
         }

         std::cout << "rank " << comm.rank() << " adding edge owners " 
<< descriptors[from].owner << " - " << descriptors[dest].owner << std::endl;
         add_edge(descriptors[from], descriptors[dest], g);
     }

     boost::synchronize(g);

     BGL_FORALL_EDGES(e, g, dgraph)
     {
         std::cout << "E @ " << comm.rank() << " " << g[boost::source(e, 
g)].s_ << " -> " << g[boost::target(e, g)].s_ << std::endl;
     }

     return 0;
}

RANK 0
RANK 1
rank 0 adding 100
rank 0 adding 101
rank 1 adding 200
rank 1 adding 201
rank 0 adding 102
rank 0 adding 103
rank 1 adding 202
rank 1 adding 203
rank 0 adding 104
rank 1 adding 204
rank 0 adding 105
rank 1 adding 205
rank 1 adding 206
rank 0 adding 106
rank 0 adding 107
rank 1 adding 207
rank 0 adding 108
rank 1 adding 208
rank 0 adding 109
rank 1 adding 209
V @ 0 100 local 0 owner 0
V @ 0 201 local 1 owner 0
V @ 0 102 local 2 owner 0
V @ 0 203 local 3 owner 0
V @ 0 104 local 4 owner 0
V @ 0 205 local 5 owner 0
V @ 1 200 local 0 owner 1
V @ 1 101 local 1 owner 1
V @ 1 202 local 2 owner 1
V @ 1 103 local 3 owner 1
V @ 1 204 local 4 owner 1
V @ 1 105 local 5 owner 1
V @ 1 206 local 6 owner 1
V @ 1 107 local 7 owner 1
V @ 1 208 local 8 owner 1
V @ 0 106 local 6 owner 0
V @ 0 207 local 7 owner 0
V @ 0 108 local 8 owner 0
V @ 0 209 local 9 owner 0
 >>> NUM VERTICES 10 NUM EDGES 0
V @ 1 109 local 9 owner 1
 >>> NUM VERTICES 10 NUM EDGES 0
rank 1 adding edge owners 0 - 1
rank 0 adding edge owners 1 - 0
Assertion failed: (v.owner == processor()), function operator[], file 
/usr/local/include/boost/graph/distributed/adjacency_list.hpp, line 1696.
Assertion failed: (v.owner == processor()), function operator[], file 
/usr/local/include/boost/graph/distributed/adjacency_list.hpp, line 1696.
E @ 0 201 -> E @ 1 101 -> [Senseis-MBP:40908] *** Process received 
signal ***
[Senseis-MBP:40908] Signal: Abort trap: 6 (6)