Re: Boost.Buffers: network-agnostic buffer types and algorithms

From: Klemens Morgenstern (klemensdavidmorgenstern_at_[hidden])
Date: 2023-01-28 04:09:21

I do like reducing dependencies, but I don't see the point of separating
things just for the sake of it.
If you're writing a boost library most users will have installed boost as a
monolith;
but even those that didn't will use this in the context of http+asio so
that an avoiding
the inclusion of `asio/buffer.hpp` doesn't really seem like a great
motivation for a new library to me.
I for one don't mind using sub-libraries, e.g. including boost/core for the
buffer types seems ok to me.

What I would however like is a unification of the two dynamic_buffer
concepts, which are currently a mess.
Taking the beast buffers and making them work nicely with asio would be
great.
But given the v1 v2 chaos,  this might not be very easy.
I would add that I think every buffer is also sink & source, so they maybe
should implement that interface as well.

Additionally, I feel some need for a library managing raw memory. It's
quite common to just use vector<unsigned char>,
but I find this incorrect, because it's typed. Raw memory doesn't have a
type,
which is why malloc returns void* and not unsigned char*.
Thus a container type for a chunk of raw memory should also be included.
I.e. something akin to a vector<void> if you will. Or like `bytes` in
python.

On Sat, Jan 28, 2023 at 3:35 AM Vinnie Falco via Boost <
boost_at_[hidden]> wrote:

> Greetings. I am asking if there is any interest in a Boost.Buffers
> library, which consists of the following:
>
> * ConstBuffer, MutableBuffer concepts
> * ConstBuffers, MutableBuffers concepts
> * const_buffer, mutable_buffer types
> * DynamicBuffer concept
> * `source` and `sink` abstract interfaces used to
>   define the Source and Sink concepts
> * metafunctions for checking all type requirements
> * implementations of common buffers and dynamic buffers:
>   - buffers_pair
>   - array_of_buffers
>   - circular_buffer
>   - dynamic_string_buffer
> * Algorithms:
>   - buffer_copy
>   - buffer_size
>
> The twist is that this library has no dependency on Asio or Boost.Asio
> (but Asio will still recognize them thanks to some template heroics
> and a splash of forward declarations). Why am I proposing this? Well,
> let me tell you some things...
>
> I am currently working on these two libraries:
>   Boost.Http.Proto
>   Boost.Http.Io
>
> The Proto library implements HTTP/1 to a fuller feature set than Beast
> and fixes all the design flaws in Beast (but it is completely not API
> compatible). This library does not do any network I/O, it is expressed
> purely in terms of buffer concepts (i.e. depends on Boost.Buffer and
> not Boost.Asio).
>
> The Io library depends on Asio and the Proto library to provide the
> network algorithms which use Asio to implement HTTP/1. This is the
> equivalent HTTP functionality of what Beast offers today.
>
> Websockets is not included but when I am done with Http I will redo
> websockets with improvements and new features, and similarly they will
> be developed as two libraries:
>
>   Boost.Websockets.Proto
>   Boost.Websockets.Io
>
> My motivation for splitting libraries this way is as follows:
>
> * Each library is smaller and compiles more quickly
> * CI turnarounds are faster
> * Separation of concerns
>
> The HTTP Protocol library is designed in a way that much of an
> application's HTTP business logic can be expressed independently of
> Asio using only the constructs found in Http.Proto. This especially
> means fewer templates and faster compilation because Http.Proto is
> designed from the ground up to avoid templates whenever possible.
>
> However during the development of Http.Proto I find myself having to
> reach for the same buffer algorithms and containers that I developed
> for Beast (such as the circular buffer and the dynamic buffer). So
> Http.Proto has been steadily growing these things:
>
> <
> https://github.com/CPPAlliance/http_proto/blob/a3a284f06597ad12ca6cc1782bebbba04d9cbf8e/include/boost/http_proto/buffer.hpp
> >
>
> Extract this independent buffer functionality into a separate library
> Boost.Buffers would bring these benefits:
>
> * Again a smaller library, faster compilation
> * Faster CI turnarounds
> * Separation of concerns
>
> Additionally, algorithms can be expressed in terms of asio-independent
> buffer types and concepts. For example, Boost.JSON could implement the
> boost::buffers::sink and boost::buffers::source interfaces which would
> enable its use in Boost.Http.Proto and Boost.Requests (an upcoming
> HTTP client library from Klemens), without having to depend on either
> of those libraries or Asio.
>
> Is this style of development of smaller, fine grained libraries that
> separate concerns something that is desirable for Boost? Glad for any
> feedback.
>
> --
> Regards,
> Vinnie
>
> Follow me on GitHub: https://github.com/vinniefalco
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>