Boost.Buffers: network-agnostic buffer types and algorithms

From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2023-01-27 19:34:51

Greetings. I am asking if there is any interest in a Boost.Buffers
library, which consists of the following:

* ConstBuffer, MutableBuffer concepts
* ConstBuffers, MutableBuffers concepts
* const_buffer, mutable_buffer types
* DynamicBuffer concept
* `source` and `sink` abstract interfaces used to
  define the Source and Sink concepts
* metafunctions for checking all type requirements
* implementations of common buffers and dynamic buffers:
  - buffers_pair
  - array_of_buffers
  - circular_buffer
  - dynamic_string_buffer
* Algorithms:
  - buffer_copy
  - buffer_size

The twist is that this library has no dependency on Asio or Boost.Asio
(but Asio will still recognize them thanks to some template heroics
and a splash of forward declarations). Why am I proposing this? Well,
let me tell you some things...

I am currently working on these two libraries:
  Boost.Http.Proto
  Boost.Http.Io

The Proto library implements HTTP/1 to a fuller feature set than Beast
and fixes all the design flaws in Beast (but it is completely not API
compatible). This library does not do any network I/O, it is expressed
purely in terms of buffer concepts (i.e. depends on Boost.Buffer and
not Boost.Asio).

The Io library depends on Asio and the Proto library to provide the
network algorithms which use Asio to implement HTTP/1. This is the
equivalent HTTP functionality of what Beast offers today.

Websockets is not included but when I am done with Http I will redo
websockets with improvements and new features, and similarly they will
be developed as two libraries:

  Boost.Websockets.Proto
  Boost.Websockets.Io

My motivation for splitting libraries this way is as follows:

* Each library is smaller and compiles more quickly
* CI turnarounds are faster
* Separation of concerns

The HTTP Protocol library is designed in a way that much of an
application's HTTP business logic can be expressed independently of
Asio using only the constructs found in Http.Proto. This especially
means fewer templates and faster compilation because Http.Proto is
designed from the ground up to avoid templates whenever possible.

However during the development of Http.Proto I find myself having to
reach for the same buffer algorithms and containers that I developed
for Beast (such as the circular buffer and the dynamic buffer). So
Http.Proto has been steadily growing these things:

<https://github.com/CPPAlliance/http_proto/blob/a3a284f06597ad12ca6cc1782bebbba04d9cbf8e/include/boost/http_proto/buffer.hpp>

Extract this independent buffer functionality into a separate library
Boost.Buffers would bring these benefits:

* Again a smaller library, faster compilation
* Faster CI turnarounds
* Separation of concerns

Additionally, algorithms can be expressed in terms of asio-independent
buffer types and concepts. For example, Boost.JSON could implement the
boost::buffers::sink and boost::buffers::source interfaces which would
enable its use in Boost.Http.Proto and Boost.Requests (an upcoming
HTTP client library from Klemens), without having to depend on either
of those libraries or Asio.

Is this style of development of smaller, fine grained libraries that
separate concerns something that is desirable for Boost? Glad for any
feedback.

-- 
Regards,
Vinnie
Follow me on GitHub: https://github.com/vinniefalco