Re: [Boost-users] [boost.asio] Concurrently Ping 1000+ Hosts

Subject: Re: [Boost-users] [boost.asio] Concurrently Ping 1000+ Hosts
From: Kyle Ketterer (reminisc3_at_[hidden])
Date: 2013-12-18 19:40:30

Gavin,

Since I will now be using one io_service object, how can I stop the threads
from blocking? When I had an io_service per object, I could just call
io_service.stop() and I could successfully call a join() on the thread.
Since I am now passing a reference to an io_service object, it seems as
though the thread will not join().

My PingX constructor is as follows:

public:
  PingX( boost::asio::io_service & io_, const char* destination,
std::vector<std::string> & ping_results, int index=0) :
ping_results_(ping_results),
  ip_addr(destination), strand_(io_), resolver_(io_), socket_(io_,
icmp::v4()),
  timer_(io_), io_service_(io_), sequence_number_(0), num_replies_(0)
  {

    socket_.non_blocking(true);
    boost::asio::socket_base::reuse_address option(true);
    socket_.set_option(option);
    _index_ = index;

    icmp::resolver::query query(icmp::v4(), ip_addr, "");
    destination_ = *resolver_.resolve(query);

    strand_.post( boost::bind( &GenericMultiPing::start_send, this) );
    strand_.post( boost::bind( &GenericMultiPing::start_receive, this) );

  }

The async functions in start_send and start_receive are strand.wrap() 'd. I
call timer.cancel() as well as socket.close() and it seems I can't get it
to unblock. Any ideas?

Main thread would be put together something like:

boost::asio::io_service io_;
boost::asio::io_service::work work_(io_);

int max_threads = 4;
int t_count = 0;

for (..i..)
{

  //buffer etc etc

  p.push_back( new XPing ( io_, buffer.data(), boost::ref(ping_results), i)
);
  threads.push_back( new boost::thread (
boost::bind(&boost::asio::io_service::run, boost::ref(io_) ) ) );

  t_count++;

  if(t_count >= max_threads) //join threads after max threads
  {
    for(...j...)
    {
      threads[j]->join();
    }

    t_count = 0;

  }

}

//cleanup threads, etc

So, I have two vectors. "p" is a vector of XPing pointers and "threads" is
where I store the threads. I also pass a ref to a separate vector which is
just a vector of ping results.

Any ideas?

On Wed, Dec 18, 2013 at 5:29 PM, Gavin Lambert <gavinl_at_[hidden]>wrote:

> On 19/12/2013 11:12, Quoth Kyle Ketterer:
>
>  In my Main thread, I was thinking about creating a vector of PingX
>> objects. In the PingX class, I would have a separate io_service as well
>> as wrap the async() functions in strands. I set the deadline timer at
>> 800ms that way the io_service should return in less than a second. I
>> would then call io_service run() in separate threads to *concurrently*
>> ping these hosts.
>>
>> The problem appears to be that even through the async handers are
>> wrapped in strands, there is data corruption. Is it true that when async
>> handlers are wrapped in strands, they still cannot be executed
>> *concurrently* ? My understanding of strand was that it serializes the
>> handler functions to allow for thread safe access. However, I think that
>> because I am accessing the same resources in multiple threads (such as
>> the socket() object ), it is corrupting my expected output as well as
>> occasionally throwing some exceptions.
>>
>
> It sounds like you're using the wrong approach.
>
> You want one single io_service object, and one io_service::strand per
> non-allowed-concurrency-task (possibly one per PingX object).
>
> You can then start up any number of threads that you wish (don't do 1000
> though -- typically maximum concurrency is achieved with somewhere between
> #CPUs - #CPUs*2 threads) and have them all call run() on the same shared
> io_service object.
>
> Any handler that is queued via the same strand object will guarantee that
> it is not executed concurrently with any other handler on the same strand
> (but has no guarantees with handlers on other strands). Additionally, even
> without strands you can guarantee that a handler cannot execute before the
> line that does the async_* operation to start it, and once called cannot
> execute again until you make another async_* call from the handler (or
> elsewhere).  Thus if your operations form a single chain (such as a read
> that is started on construction and continued only at the end of the
> handler) then those reads can't execute concurrently with themselves even
> without a strand.  (But you still might want a strand or lock to protect
> data shared between read and write, or between multiple concurrent writes,
> or separate objects with shared data.)
>
> Note that if you want the "ping"s to occur concurrently you'll have to use
> different sockets for each one.  Up to you whether you create one socket
> for each PingX or make a smaller pool that is shared out as needed (but the
> former is easier, and the second will need thread protection on the pool).
>
>
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>

-- 
Kyle Ketterer
<reminisc3_at_[hidden]>
215-208-8523