Re: [Boost-bugs] [Boost C++ Libraries] #3150: boost.asio: async_copy(in, out, ...) for POSIX sendfile() support

Subject: Re: [Boost-bugs] [Boost C++ Libraries] #3150: boost.asio: async_copy(in, out, ...) for POSIX sendfile() support
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2009-08-11 09:43:27

#3150: boost.asio: async_copy(in, out, ...) for POSIX sendfile() support
---------------------------------+------------------------------------------
 Reporter:  trapni@…             |       Owner:  chris_kohlhoff
     Type:  Feature Requests     |      Status:  new           
Milestone:  To Be Determined     |   Component:  asio          
  Version:  Boost 1.39.0         |    Severity:  Optimization  
 Keywords:  sendfile posix asio  |  
---------------------------------+------------------------------------------
Changes (by Christian Parpart <trapni@…>):

 * cc: trapni@… (added)

Comment:

 I quite don't think that this has anything to do with HA (::= High
 Availability) but much more with "High Performance" applications.

 However, I'm really very interested in a native boost integration of this,
 too, but still, i think this can be quite challenging as the input and
 output descriptors need to match certain criteria on different UNIX-alike
 operating systems (btw: windows has something similar)

 for Linux at least, the requirements are:
  * input fd must be mmap()-able.
  * output fd must be a socket.

 Although, when it comes to implementing an HTTP server (in my case), you
 cannot just pass the filename as a string to async_copy(...) as there are
 moments where you want to send just parts of a file, or even further:
 multiple different ranges of a file with some memory "chunks" between.

 Although, as in HTTP (and I assume this case for most of the file transfer
 related network protocols) before sending the plain file, you always send
 some kind of preamble/header, some memory chunk that introduces the range
 of bytes located in the file.
 Solaris helps out here with a special syscall, named sendfilev(), which is
 a combination of writev() and sendfile().

 In the end, i think, that a simple "async_copy()" won't be able to cover
 all this, although, when just implementing a sendfile() wrapper, it won't
 be long to the next request of sendfilev().

 To solve that, I'd like to propose a generalized idea of `composed
 transmit buffers` which is an ordered set of chunks of different types
 (e.g.: mem, file_descriptor, ...) which can be passed to some write(...)
 and async_write(...) Asio function.

 This is basically what I have written for my http server case, i'd be
 happy to see something like this in Asio anytime soon :)

 The following is a list of links where I implemented the composite
 transmit buffers with sendfile() support:
  *
 http://code.ninchens.net/projects/x0/repository/entry/src/x0/composite_buffer.hpp

  *
 http://code.ninchens.net/projects/x0/repository/entry/src/x0/composite_buffer_async_writer.hpp

  *
 http://code.ninchens.net/projects/x0/repository/entry/src/x0/composite_buffer_writer.hpp


 hope it'll inspire asio development process :)

-- 
Ticket URL: <https://svn.boost.org/trac/boost/ticket/3150#comment:2>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.