Re: [Boost-users] asio: how to handle disk IO, recommendations

Subject: Re: [Boost-users] asio: how to handle disk IO, recommendations
From: Acer Yang (yangacer_at_[hidden])
Date: 2012-10-23 22:40:14

On Wed, Oct 24, 2012 at 12:50 AM, Avi Bahra <avibahra_at_[hidden]> wrote:

> > Currently I has a single threaded server based on boost ASIO.
> > This works correctly most of the time.
> > However on systems where i know that the disk's are slow.
> > , the server grinds to a halt, when dealing with the blocking
> > disk IO.
> >
> > I could not find any examples or documentation in ASIO, in
> > how to get round these issues. It appears that ASIO does
> > not provide any support for async' disk IO.
> >
> > So I assume this is handled by use of threads ?
> > Are there any examples of using ASIO with threaded disk IO,
> > since this must be very common use case ?
> >
> > Is the strand functionality appropriate for handing disk IO,
> > in an async' manner.
> >
> > Any recommendations/advice on this issues would be
> > greatly appreciated.
> >
> > Best regards,
> > Ta,
> >     Avi
> > _______________________________________________
> > Boost-users mailing list
> > Boost-users_at_[hidden]
> > http://lists.boost.org/mailman/listinfo.cgi/boost-users
> >
>
> >Hi Avi,
> >
> >If your server performs short read/writ frequently and data to be transfer
> >to network depend on these operations, then multi-threading does not help
> a
> >lot here. I would suggest you to recheck your I/O strategies and make sure
> >they are optimized. e.g. buffering, cache, .. etc. If they are perfect but
> >performance still bother you, then consider to use memory mapped file or
> >libaio.
> >
> >Best,
> >Acer.
>
> In my  case the ASIO server needs to only write to a local disk.(no reads)
> One write is relatively short, i.e to a log file, to record all requests,
> the other is to write periodically a in memory node tree.
> This can vary between 1-100MB.
> My thinking was that if I used a thread reserved for writing the
> node tree, it will not block the server from handling requests,
>  on slow file systems.
>
> I will look into memory mapped files and libaio.
>
> Ta, Avi
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>

Log is a good candidate to be improved. You can look into asio's document.

Here is an online version
http://www.boost.org/doc/libs/1_51_0/doc/html/boost_asio/examples.html

These documents are included in every boost distribution.

The 'Services' section involves a simple logger service which can perform
write
operation in a background thread. Furthermore, you can also wrap your large
write operation as a service to achieve shorter response time.

Best,
Acer.