Re: [Boost-users] [boost.asio] Concurrently Ping 1000+ Hosts

Subject: Re: [Boost-users] [boost.asio] Concurrently Ping 1000+ Hosts
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2013-12-20 19:53:23

On 20 Dec 2013 at 19:16, Bjorn Reese wrote:

> > Can you suggest something? I honestly can't think of anything simpler
> > which also provides strong write ordering guarantees.
> 
> I have not given this much thought so consider the following a
> brainstorm.

We really ought to move this off boost-users ... but we'll see how it 
goes.

> I am thinking about an API that uses handles that looks more like Asio 
> sockets. Write ordering can be handled, not by batching operations
> together, but rather calling the next write operation from the callback
> of the previous operation (Asio-style.) This will not always yield good
> performance, but oftentimes that is less relevant. If you need
> performance, then the "advanced" dispatcher API is available.
> 
> So there could be a file handle (and directory handle) class for file
> (directory) manipulation calls, which hides all the details of the
> dispatcher etc.
> 
>    class file_handle
>    {
>    public:
>      void read(buffer, read_callback);
>      void write(buffer, write_callback);
>      // and so on
>    };
> 
>    class directory_handle
>    {
>    public:
>      void create(name, create_callback); // uses file(single) or dir(single)
>      void remove(name, remove_callback); // uses rmdir(single)
>      void watch(name, watch_callback); // directory monitoring
>      // and so on
>    };

I'm struggling to see the merit in such an approach - it would be 
incredibly verbose and complex to write even simple solutions, 
because i/o on files is not like sockets, *especially* that you 
almost never must strongly order i/o across a sequence of multiple 
sockets, but that is a very common case with files e.g. during ACID. 
AFIO tries to help users to not write callbacks except when 
necessary, but if you do want a user defined callback you simply 
chain a call() or completion() operation to the item whose completion 
you want called back upon.

The idea is, you see, that you subclass the async_io_dispatcher class 
with additional completion handlers, and use those as building blocks 
for further subclasses of async_io_dispatcher. That hopefully gets 
people to break up their callbacks into reusable completion handlers, 
and saves people writing and debugging code.

Before you say "this should be in the documentation", yes it should 
and will be after Paul gets his directory monitoring implementation 
working. I'm thinking that will form the fourth section in the 
beginner's tutorial - how to modularise and make reusable the 
normally bespoke glue code which makes up traditional callbacks. It 
might actually be worth adding a section 3b which shows how the naïve 
approach in 3a is a very stupid idea :)

As much as this sort of operation dependency graph based design works 
well for its niche, I agree there is a swathe of code which ends up 
looking like the find in files implementation, and for that hopefully 
fibers will make look much more sane.

Niall

-- 
Currently unemployed and looking for work.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/