[boost] [asio] Draft version uploaded to vault

From: Christopher Kohlhoff (chris_at_[hidden])
Date: 2006-04-26 03:21:24

Hello all,

I have just uploaded a draft version of Boost.Asio to the vault
(in the "Input - Output" folder). I'm particularly interested in
feedback on the handler-based custom memory allocation and
IPv4/v6 protocol independence.

Please note: this is not a properly tested release. Caveat
emptor. It has the following known issues:

- It uses getaddrinfo/getnameinfo, but these functions are not
  thread-safe on Mac OS X. They will be replaced by thread-safe
  emulation using getipnodebyname and getipnodebyaddr.

- Although builds using MSVC or Borland C++ automatically
  emulate getaddrinfo/getnameinfo on Windows 2000 and earlier,
  MinGW does not. Therefore MinGW currently only supports
  Windows XP and Windows Server 2003 targets.

- It has not been compiled or tested at all on Solaris. It has
  been tested on Mac OS X 10.4, Debian Sarge with 2.6 linux
  kernel and Windows XP.

Cheers,
Chris

Changes since 0.3.6
===================

(Excluding minor bug fixes and in no particular order.)

* Renamed demuxer to io_service.

* All services inherit from a common base io_service::service.

  As part of this change, the demuxer::get_service() member
  template function has been replaced with three free template
  functions: use_service(), add_service() and has_service().

* Resource classes inherit from basic_io_object.

  Sockets, timers, etc inherit from basic_io_object<Service>.
  This class automatically obtains the correct service object
  from the io_service, and provides the typedefs and functions
  that all objects must provide (e.g. an io_service() member
  function for obtaining the associated io_service object).

* All handler objects destroyed during io_service destruction.

  Previously, if you did not allow demuxer::run() to continue
  until it had no more work to do, and then destroyed the
  demuxer, uninvoked handlers would never be destroyed.

  All stored handler objects that are "owned" by the io_service
  or any associated services are now destroyed automatically
  from inside the io_service. This allows quick, clean shutdown
  of programs using io_service::interrupt().

  This handler cleanup has been implemented using "two-phase"
  service shutdown. Every service that inherits from
  io_service::service must implement a virtual function called
  shutdown_service(). The service's implementation of this
  function is required to destroy all copies of handlers that it
  owns (directly or indirectly). The cleanup must be done in a
  separate phase prior to the destruction of the service
  objects, since the handler destructors may try to access other
  services.

* Renamed locking_dispatcher to strand.

  A strand is defined as a strictly sequential invocation of
  event handlers (i.e. no concurrent invocation). Use of strands
  allows execution of code in a multithreaded program without
  the need for explicit locking (e.g. using mutexes).
  
  Strands may be either implicit or explicit, as illustrated by
  the following alternative approaches:

    - Calling io_service::run() from only one thread means all
      event handlers execute in an implicit strand, due to the
      io_service's guarantee that handlers are only invoked from
      inside run().

    - Where there is a single chain of asynchronous operations
      associated with a connection (e.g. in a half duplex
      protocol implementation like HTTP) there is no possibility
      of concurrent execution of the handlers. This is an
      implicit strand.

    - An explicit strand is an instance of asio::strand. All
      event handler function objects need to be wrapped using
      asio::strand::wrap() or otherwise posted/dispatched
      through the asio::strand object.

* Add macros to disable platform-specific implementations.

  The following macros can be used to disable platform-specific
  implementation code and revert to portable select-based
  asynchronous I/O emulation:

    - BOOST_ASIO_DISABLE_EPOLL
    - BOOST_ASIO_DISABLE_IOCP
    - BOOST_ASIO_DISABLE_KQUEUE

* Handler-based custom memory allocation.

  Many asynchronous operations need to allocate an object to
  store state associated with the operation. For example, the
  Win32 implementation needs OVERLAPPED-derived objects to pass
  to Win32 API functions.
  
  Furthermore, programs typically contain easily identifiable
  chains of asynchronous operations. A half duplex protocol
  implementation (e.g. an HTTP server) would have a single chain
  of operations per client (receives followed by sends). A full
  duplex protocol implementation would have two chains executing
  in parallel. Programs should be able to leverage this
  knowledge to reuse memory for all asynchronous operations in a
  chain.

  Given a copy of a user-defined Handler object h, if asio needs
  to allocate memory associated with that handler it will
  execute the code:

    void* pointer = asio_handler_allocate(size, &h);

  Similarly, to deallocate the memory it will execute:

    asio_handler_deallocate(pointer, &h);

  These functions are located using ADL. The asio library
  provides default implementations of the above functions in the
  global namespace:

    void* asio_handler_allocate(size_t, ...);
    void asio_handler_deallocate(void*, ...);

  which are implemented in terms of ::operator new() and
  ::operator delete() respectively.

  The implementation guarantees that the deallocation will occur
  before the associated handler is invoked, which means the
  memory is ready to be reused for any new asynchronous
  operations started by the handler.

  Custom memory allocation support is not yet used throughout
  the asio implementation. In particular, the reactor-based
  emulation still uses dynamically allocated objects. It is
  fully implemented for the Win32-specific I/O completion port
  code. The custom memory allocation support has also been
  disabled for Borland C++, since it generated obscure compiler
  errors.

  See libs/asio/example/allocation/server.cpp for an example of
  how to use the handler-based custom memory allocation.

* Allocator template parameter removed.

  Removing the Allocator template parameter reduces the number
  of core types in asio and makes it easier to develop custom
  services (i.e. you don't have to worry what Allocator object
  the io_service has been templated on). It should also make it
  simpler to create a shared or static library-based
  implementation.

  A combination of handler-based custom memory allocation, and
  the ability for the asio implementation to embed objects
  directly into object internals (e.g. socket implementations),
  should minimise dynamic memory allocations.
  
  If there is a demonstrable need, I may consider adding an
  io_service::service-based allocator to allow memory allocation
  to be further customised on a per-io_service-object basis.
  
* Add support for building without thread support.

  POSIX platforms only. The Windows implementation still
  requires threads.

* Async waits cancelled by deadline_timer expires_*() setters.

  Outstanding asynchronous wait operations are now automatically
  cancelled if you call expires_at() or expires_from_now() to
  modify the expiry time of a deadline_timer. The previous
  documented behaviour was that it was "undefined" to modify the
  expiry time while there were outstanding waits.

* Support for reference-counted buffers.

  The Const_Buffers and Mutable_Buffers concepts have been
  modified to allow reference counted buffers to be used. In
  particular:

    - The {Const|Mutable}_Buffers::iterator typedef and
      non-const begin()/end() functions have been removed from
      the concepts' requirements. Implementations of the
      concepts now need only provide const_iterator support, and
      do not need to allow in-place modification of the
      elements.

    - The asio implementation guarantees that at least one copy
      of the buffers object will be maintained for as long as
      the system may need access to the data.

    - The asio implementation explicitly converts the
      dereferenced const_iterator into mutable_buffer or
      const_buffer as needed.

  See the libs/asio/example/buffers/reference_counted.cpp
  example program.

* Strongly-typed socket classes.

  Rather than asio::stream_socket, asio::datagram_socket and
  asio::socket_acceptor classes that were used for all
  protocols, these socket classes are now specific to each
  protocol. For example, we now have:

    ip::tcp::socket
    ip::tcp::acceptor
    ip::udp::socket

  and in the future we might have:

    unix::stream_protocol::socket
    bluetooth::rfcomm::socket

  and so on. These protocol-specific classes are simply typedefs
  for templates instantiated on the corresponding Protocol
  class. E.g.:

    template <typename Protocol>
    class stream_socket_service;

    template <typename Service>
    class basic_stream_socket;

    class tcp
    {
      ...
      typedef stream_socket_service<tcp> socket_service;
      typedef basic_stream_socket<socket_service> socket;
      ...
    };

  New protocols can be added by creating classes like tcp above.

* Use inheritance in socket types to prevent "layer violations".

  The basic_stream_socket and basic_datagram_socket templates
  now inherit from basic_socket<Service>. This class includes
  all functions that are common to stream and datagram sockets
  (everything except send/receive and friends). Stream layers
  such as ssl::stream return a reference to basic_socket<> from
  the lowest_layer() function.

* IPv4 and IPv6 protocol independence.

  Socket, endpoint and address classes for IP (i.e. TCP and UDP)
  are independent of the particular IP version (4 or 6) used.
  The ipv4 namespace has been removed. For example:

    - The ipv4::address class has become IP version-independent
      ip::address. If you need to manipulate addresses for a
      specific IP version use ip::address_v4 or ip::address_v6.

    - The protocol classes are now ip::tcp and ip::udp, and the
      corresponding endpoint types are ip::tcp::endpoint and
      ip::udp::endpoint.

    - To open a socket you must specify the protocol version,
      e.g.:

        ip::tcp::socket socket(io_service);
        ...
        socket.open(ip::tcp::v4()); // IPv4
        ...
        socket.open(ip::tcp::v6()); // IPv6

      Note: if you already have an endpoint you can avoid this
      by using the protocol associated with the endpoint object:

        socket.open(endpoint.protocol());

    - When constructing an endpoint with just a port number, you
      must specify the IP version:

        ip::tcp::endpoint(ip::tcp::v4(), 12345);

  The development of programs that are independent of the IP
  version is further aided by the new resolver interface
  outlined below.

* Resolver replaces ipv4::host_resolver.

  The ipv4::host_resolver and ipv4::host classes have been
  removed in favour of an endpoint-oriented interface that
  can support different protocols. The functionality of this new
  interface is based on getaddrinfo and getnameinfo.

  Use the resolver by constructing a query, calling resolve()
  (or async_resolve()), and using the returned forward-only
  iterator to enumerate the endpoints that match the query. For
  example:

    ip::tcp::resolver resolver(io_service);
    ip::tcp::resolver::query query("foobar.com", "http");
    ip::tcp::resolver::iterator iter = resolver.resolve(query);
    ip::tcp::resolver::iterator end;
    while (iter != end)
    {
      ip::tcp::endpoint ep = *iter;
      std::cout << iter->endpoint() << std::endl;
      std::cout << iter->host_name() << std::endl;
      std::cout << iter->service_name() << std::endl;
      ++iter;
    }

  The asynchronous equivalent is:

    void my_class::start_resolve()
    {
      ip::tcp::resolver resolver(io_service);
      ip::tcp::resolver::query query("foobar.com", "http");
      resolver.async_resolve(query,
          boost::bind(&my_class::resolve_handler, this,
            asio::placeholders::error,
            asio::placeholders::iterator));
    }

    void my_class::resolve_handler(const asio::error& e,
        ip::tcp::resolver::iterator iter)
    {
      if (!e)
      {
        ip::tcp::resolver::iterator end;
        while (iter != end)
        {
          ...
          ++iter;
        }
      }
    }

  There are overloads of resolve()/async_resolve() that take an
  endpoint and return an iterator. These are intended for
  reverse lookups.

  Since the ipv4::host_resolver had a local_host() function that
  returned information about the local system, a new function
  ip::host_name() has been added which returns a string
  containing the name of the system.

* Multicast socket options renamed.

  The IP-specific socket options for multicast have been
  renamed. They are now also independent of the IP version. The
  name changes are:

    - add_membership is now join_group
    - drop_membership is now leave_group
    - time_to_live is now hops

  These multicast socket options are now found in the
  ip::multicast namespace.

* String-to-address conversion now uses from_string function.

  This change was driven by three things:

    - The need to indicate an error without throwing if a
      conversion from string to address fails.

    - String literal addresses would not be common and so
      support for them does not need to be convenient.

    - It is more common to use a resolver to convert strings
      into complete endpoints.

  To convert a string into an address you would now write:

    ip::address address = ip::address::from_string("1.2.3.4");

  which throws an asio::error on failure, or:

    asio::error error;
    ip::address address = ip::address::from_string(
        "1.2.3.4", asio::assign_error(error));

  to convert without throwing. Similar functions exist on the
  ip::address_v4 and ip::address_v6 classes.

* IPv4 and IPv6 addresses can be converted to and from bytes.

  The ip::address_v4 and ip::address_v6 classes contain:

    - A typedef called bytes_type, which is a
      boost::array<unsigned char, N>. For IPv4, N is 4. For
      IPv6, N is 16.

    - A constructor that takes a bytes_type value.

    - A function to_bytes() that returns a bytes_type value.

* Order of endpoint constructor params reversed.

  The old ipv4::tcp::endpoint class (and udp equivalent) had a
  constructor that took a port and address in that order. The
  new ip::tcp::endpoint and ip::udp::endpoint classes have the
  order reversed, since that is more idiomatic.

* Change position of flags param on send_to, receive_from.

  The order of parameters on the basic_datagram_socket's
  send_to, async_send_to, receive_from and async_receive_from
  functions have been changed to put the endpoint before the
  message flags.
  
  Overloads of these functions have been added that do not
  include the flags parameter, which gives the flags a default
  value of 0. A similar change has been made for the send,
  async_send, receive and async_receive functions on
  basic_datagram_socket and basic_stream_socket.

* Don't globally block SIGPIPE if an alternative is available.

  On Linux, the MSG_NOSIGNAL flag is now passed to the sendmsg
  function. On Mac OS X, the SOL_SOCKET/SO_NOSIGPIPE socket
  option is set on all sockets when they are created. The
  SIGPIPE signal is globally ignored on Solaris.

* Reactor implementations run operations until complete.

  The reactor socket implementation sets non-blocking I/O for
  asynchronous emulation (non-blocking mode is enabled for a
  socket the first time an async call is made on that socket).
  Operations are now restarted automatically if the asynchronous
  operation returns would_block or try_again, meaning that
  asynchronous operations are guaranteed never to return these
  errors.

Changes planned for 0.3.7
=========================

* Fix tutorials/examples to not use "C style" code.

* Emulation of getaddrinfo/getnameinfo on platforms where it is
  unavailable or broken.

Changes planned for post-0.3.7
==============================

* Iostreams support.

* Split_error handler function object.

* Overloads of io_service::run() that take a timeout parameter.

  This is to allow calls to run() to be interleaved with other
  application code. The run() function will return an enum value
  indicating why it finished (e.g. timed_out, out_of_work,
  interrupted, etc).

* Polymorphic wrapper for the Dispatcher concept.

* Polymorphic wrapper for the Stream concept.

* Investigate handler-invocation hooking.

* Develop a performance test framework.

* Rework reactor implementations to support handler-based custom
  memory allocation.

* New reactor implementations, e.g. /dev/poll and Solaris 10
  ports.

* Use the Windows API call ConnectEx where available for true
  asynchronous socket connect operations.

* Add better support for line-based protocols.

  This might take the form of an {async_}read_line function or
  equivalent.

* Add UNIX domain socket support.

* Documentation, documentation, documentation.

I hope that's everything!