Re: Boost.Buffers: network-agnostic buffer types and algorithms

From: Andrzej Krzemienski (akrzemi1_at_[hidden])
Date: 2023-02-04 14:42:13

pt., 3 lut 2023 o 02:02 Vinnie Falco <vinnie.falco_at_[hidden]> napisał(a):

> On Thu, Feb 2, 2023 at 1:58 PM Andrzej Krzemienski <akrzemi1_at_[hidden]>
> wrote:
> > Today the basics for networking seems like good candidates for
> > vocabulary types. Is Boost.Buffer a candidate for such a vocabulary type?
>
> In theory, yes. The intent is for Buffers to be more general than just
> networking.
>
> > First question, is this library to be consumed directly by user programs,
> > or only by other libraries, such as Boost.Beast and Boost.Http.Proto?
>
> User programs could use it when they want to express algorithms in
> terms of ranges of buffers. My personal use-case is Beast, HTTP,
> Websocket, and JSON but this could be extended to say, base64 encoding
> and decoding, calculating SHA-2 digest over a range of buffers,
> simplified front end to things like zlib. Example:
>
>     template<
>         class DynamicBufferOut,
>         class DynamicBufferIn >
>     result< void >
>     deflate(
>         z_params&,
>         DynamicBufferOut&
>         DynamicBufferIn&,
>         bool end_of_data);
>
> > Is it possible that the final program that uses Boost.Buffer will not
> use Boost.ASIO?
>
> That is possible, yes.
>
> > For me, as a consumer of networking libraries, who can afford to
> > download and build the entire set of Boost libraries, the biggest
> > advantage would be that Buffers would be well documented and tested.
>
> That is an advantage yes but in my opinion the advantage is that you
> get access to a rich and useful set of algorithms, containers, and
> concepts, without also depending on a particular network
> implementation.
>
> > I am not convinced by the argument with Boost.JSON not depending on ASIO.
>
> Well, users definitely do not want to drag in Asio with their JSON.
>
> > We have (I think a still unresolved issue) of Boost-serializing
> Boost.Optional.
> > Should the code responsible for the serialization of Boost.Optional be
> part of
> > Boost.Serialization or Boost.Optional? But we will certainly not solve
> it by
> > introducing yet another library.
>
> I think that the cases of Boost.Serialize with boost::optional, and
> serialization of JSON using Boost.JSON and Boost.Buffers are two
> completely different cases. Boost.Serialize does not define a
> serialization format but rather a framework for allowing the user to
> define serialization. Boost.JSON is something else entirely; it is a
> library which implements a specific protocol for describing data using
> structured text.
>
> If you look at Boost.JSON you see that it has json::parser and
> json::serializer. These classes work a buffer at a time and they
> support streaming; that is, that the entire input or output need not
> be presented at once in order to make progress. How does Boost.Buffers
> help here? Well.. it provides the means for a library such as JSON,
> which has algorithms to produce or consume a buffer at a time, to
> type-erase its algorithm using the buffers::source or buffers::sink
> interfaces:
>
>     /// An algorithm which consumes filled buffers
>     struct  sink
>     {
>         virtual ~sink() = 0;
>
>         /** Called when data is available
>         */
>         virtual result<void> write( span< const_buffer const > ) = 0;
>     };
>
>     /// An algorithm which produces filled buffers
>     struct source
>     {
>         /** The results of reading from the source.
>         */
>         struct results
>         {
>             error_code ec;
>             std::size_t bytes = 0;
>             bool more = false;
>         };
>
>         virtual ~source() = 0;
>
>         /** Called when more data is required
>
>             This function is invoked when more data
>             is required. The subclass should place
>             zero or more bytes into the buffers
>             referenced by `dest`, and return a
>             `results` value with the members set
>             appropriately.
>             <br>
>             Partial success is possible.
>         */
>         virtual results read( span< mutable_buffer const > dest ) = 0;
>     };
>
> This is not really any different from libraries that support
> operator<<(std::ostream&), except that the vocabulary type is
> buffers::sink and buffers::source instead of std::istream and
> std::ostream. When Boost.JSON implements these APIs, it is not
> "supporting Boost.Serialization" it is merely providing a wrapper for
> the algorithms it already has (json::parser and json::serializer) to
> adapt them to a common, agreed-upon interface (the buffers::sink and
> the buffers::source).
>
> Currently users complain that Boost is a huge bundle of libraries, and
> at some level this is true. But it is a bundle of libraries for a
> reason, because releasing the collection together as a unit allows us
> to ensure that all the components work together. But more importantly
> it allows a member library to leverage the advantages of having other
> member libraries available to it (Config, Variant2, mp11 come to
> mind). The point of implementing Boost.Buffers' sink and source
> interfaces in JSON is to let us leverage the benefits of the
> monolithic distribution model.
>
> > Boost.JSON doesn't need to depend on the implementation of the circular
> buffer.
>
> Right, it wouldn't. It would only be implementing the sink and source
> abstract interfaces. This let's JSON participate in the growing
> ecosystem of Boost.Buffers enabled algorithms. I think JSON is the
> place for it, because the model can scale. Candidate libraries which
> would add value by implementing source and sink do so in the library
> itself. The model where a single library has to depend on an ever
> increasing number of other libraries in order to implement their
> wrappers is unsustainable.
>

Thank you. I am convinced that extracting the Boost.Buffers library in the
way you described is a good idea.

Regards,
&rzej;

>
> --
> Regards,
> Vinnie
>
> Follow me on GitHub: https://github.com/vinniefalco
>