Re: Jason is getting MARRIED!

From: Lee Clagett (forum_at_[hidden])
Date: 2019-11-23 18:18:59

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Sunday, November 17, 2019 6:31 PM, Robert Ramey via Boost <boost_at_[hidden]> wrote:

> On 11/17/19 8:57 AM, Peter Dimov via Boost wrote:
>
> > Robert Ramey wrote:
> >
> > > Implementing JSON for boost serialization wouldn't be very hard. I'm
> > > surprised that in 15? years no one has done it.
> >
> > Your second sentence should make you at least question the first one.
> > The problem is as I already outlined; if you want to support field
> > reordering,
>
> serialization doesn't require reordering. the json reader for
> serialization only needs to be able to read archives created by the json
> writer for serialization which specifies the order.
>
> > If you don't support field reordering, the format would basically only
> > interoperate with itself, which defeats the purpose of using JSON. You
> > might as well just use the text archive.
>
> Exactly. Which is probably why no one ever bothered to write a JSON
> archive. Which is exactly the reason that writing an xml version was an
> unnecessary waste of time.
>
> The idea that it's possible to make an "editable archive" with XML or
> JSON or anything else is fundamentally false. There is no way that one
> could map in the general case an "edited" archive to a pre-determined
> C++ data structure. Of course one could conjure up some specific cases
> where it would be conceivable - but then we be in the C++ committee trap
> of engaging in a fools errand of trying to implement a general idea by
> specifying a bunch of special cases. Nothing would be served by going
> there.
>
> If someone wants to make a JSON version of boost serialization that's
> fine. But don't think that you can make an implementation which is
> independent of the C++ data structures being serialized.
>
> Look to a different model. Ideally one would have parse JSON to some
> general C++ format which one can then pick through and retrieve what he
> wants or determine that it's not in there. Another way to think about
> it is to replace google protocol buffers. The later requires that you
> make a separate structural syntax which is a pain. But protocol buffers
> is even more popular than boost serialization. So I think a Boost JSON
> parser would success.

This entire section highlights my frustrations with the JSON format and
most C++ JSON implementations. The bulk of cases are JSON -> specific
data structure. The easiest implementation for JSON objects in C++ is
storing all fields to a temporary DOM and then doing a lookup when
mapping to a data structure. I wrote an implementation for msgpack (not
fully tested :/) that uses C++ template variadics to skip the DOM step;
almost no one is inspecting arbitrary fields and a linear search across
a hard-coded number of fields in a C++ struct is typically quicker than
dynamically managing some kind of map. And if you don't mind punishing
the compiler, I think its possible to sort the fields automagically in
C++14 constexpr for logarithmic lookup.

Boost.Serialization cannot support this DOM-less mode without an
interface change unfortunately, and I'm not sure if this type of
interface is appropriate for Boost. A quick glance at this proposed
library suggests that it should be possible to write a new parser with
a different interface that leverage the same backend parsing code
(something which should be possible with any SAX implementation I
think).

> My personal requirements for such a system would be:
>
> a) ability to handle unlimited/unterminated data. As data is read in -
> an event is triggered when a grammatical element is recognized.
>
> b) Events are implemented by the users (though the library would provide
> a bunch of defaults). This provides for infinite flexibility, parallel
> execution for large datasets.
>
> c) Of course such a system could be implemented in a direct, verifiably
> correct manner with boost spirit. I don't know what the performance
> implications would be. Boost XML de-serialization has been done by
> spirit (first version) for 15 years and no one has complained about the
> speed. Whenever anyone complains about the speed of text based archives
> it always goes back to the file system - which presumably would be the
> case with any JSON library.
>
> Robert Ramey

Lee