Re: Jason is getting MARRIED!

From: Vinícius dos Santos Oliveira (vini.ipsmaker_at_[hidden])
Date: 2019-11-25 16:02:05

Em dom., 24 de nov. de 2019 Ã s 12:30, Vinnie Falco via Boost <
boost_at_[hidden]> escreveu:

> I'm not seeing where trial.protocol has incremental algorithms,
> perhaps you can show me? trial::protocol::json::basic_reader
> constructs with the complete input:
>
> <
> https://github.com/breese/trial.protocol/blob/4bdf90747944f24b61aa9dbde92d8f6dd6758c94/include/trial/protocol/json/reader.hpp#L49
> >
>
> There is no API to provide additional buffers. By "incremental" I mean
> an "online algorithm", i.e. the entire input does not need to be
> presented at once. For example, this is what it might look like using
> boost.json to incrementally parse a JSON from a socket:
>
>     json::value parse( net::ip::tcp::socket& sock )
>     {
>         error_code ec;
>         json::parser p;
>         p.start();
>         for(;;)
>         {
>             char buf[4096];
>             auto const n = sock.read_some(
>                 net::mutable_buffer(buf, sizeof(buf)), ec);
>             if(ec == net::error::eof)
>                 break;
>             if(ec)
>                 throw system_error(ec);
>             p.write(buf, n, ec);
>             if(ec)
>                 throw system_error(ec);
>         }
>         p.finish();
>         return p.release();
>     }
>
> Serialization functions similarly. The caller provides a buffer, and
> the implementation attempts to fill the buffer with serialized JSON.
> If the buffer is not big enough, subsequent calls may be made to
> retrieve the rest of the serialized output.
>

There are a few remarks that ought to be added to these statements.

What I have in mind for an incremental parser is an in situ/in-place
algorithm in which case there is no auxiliary data structure (i.e. only a
small and constant amount of space may be used). These are my expectations
when I see a library which advertises incremental parsing.

I don't expect the library to internally buffer everything I feed it to if
it advertises itself as an incremental parser. Yet, this is what I see in
your example itself, which is using the DOM object itself as a buffer:
https://github.com/vinniefalco/json/blob/04fe8c2ba8c3414e51a44017638688063e1caced/include/boost/json/parser.hpp#L83

This is from your example, not from the library. The library at least
offers basic_parser which does meet my expectations (for an incremental
parser). JSON allows recursion and it is not really possible to parse
arbitrary JSON values w/o at least a stack of states which both
Trial.Protocol and your library use. So even if they aren't strictly in
situ parsers either, this unavoidable violation is acceptable.

And now back to your expectation, there are other concerns that I want to
bring to the table. The property you want to offer is useful to streamable
formats, but JSON has never been a streamable format. No libraries out
there offer such functionality (please correct me if I'm wrong) and
therefore, as an interchange format for a multitude of services, the
messages designed around JSON have not been designed to rely on the
property to stream any information. If streaming is required, you're better
off with an extra communication channel to transfer the streamable data.

The context switch to parse small chunks of independent JSON values would
actually hurt the cache.

With all that in mind, even if streamable JSONs was a property we wanted to
tackle, the design of Trial.Protocol can easily accommodate such feature
with very small changes. My HTTP (which is a streamable format) parser was
inspired by Trial.Protocol and this was the first divergence I had to
tackle (which was pretty easy to solve).

-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/