Re: [review] [json] My JSON review

From: Mathias Gaunard (mathias.gaunard_at_[hidden])
Date: 2020-09-21 15:25:52

On Mon, 21 Sep 2020 at 15:23, Vinnie Falco <vinnie.falco_at_[hidden]> wrote:
>
> "JSON-inspired binary formats" are not JSON. CBOR for example is
> "JSON-inspired", yet neither Node.js nor any web browser support it.
> They do however support standard JSON, and their respective DOMs have
> numerical limitations similar to the ones in Boost.JSON.

The main reason I suggested this is that you might also want to be
able to parse your in-memory representation as some kind of visitation
mechanism which you could push directly to the serializer.
You cannot since conversion of numbers to binary occurs, so there is
no string left.
So I'd design the parser callbacks so that numbers can be either
stored as text or binary in the source.

> I do not think it is possible for a single parser API and single
> serializer API to address both of these use-cases well. Optimizing for
> one will necessarily come at the expense of the other. Perhaps there
> is a hypothetical not-yet-written library just waiting to be
> discovered which will prove me wrong, by doing everything well. I have
> my doubts.

Doing a pull parser that supports data being pushed incrementally is
difficult as it requires the ability to suspend arbitrary code that is
trying to pull data beyond what is available.
That would be doable, maybe with a coroutine design, but adds a lot of
complexity.

To make people happy I think it's easier to provide a separate
incremental push parser and a non-incremental pull parser.
I don't think those should be in separate libraries, consistency is key.

> > The DOM does not support numbers that are neither 64-bit integral
> > numbers nor double-precision floating-point. This is a common
> > limitation, but still quite disappointing, since the proposal of it
> > being a canonical type is quite tainted by the fact it can't represent
> > JSON data losslessly. I think it would be better if another type was
> > provided for numbers that don't fit in the previous two categories.
>
> I disagree. Serialization of a canonical type should produce JSON that
> is interoperable with the majority of receivers. Neither Node.js nor
> browser implementations of JSON understand arbitrary precision
> integers out of the box. Unless the receiver is configured to process
> it, emitting an arbitrary precision number is certain to cause a
> compatibility problem.

You are writing the receiver, it is the JSON parser. Making it
correctly deal with numbers is important so that you don't corrupt the
work of whatever you are consuming.
If you know you are writing data for a consumer with certain
limitations, then up to you to transform your numbers into something
else.
Note that your code will still happily write 64-bit integers as
numbers, even though JavaScript can't handle more than 53 bits, and
will round those numbers and cause all sorts of problems.

Another thing is that I don't buy the argument that web browsers,
Node.JS and all the JavaScript ecosystem are the only thing that is
relevant.
The web is a domain that has a reputation of not being quite as
rigorous as systems programming which is the main application domain
as C++.
It is ok if script kiddies want to do whatever with their webby stuff,
but let's please do things properly when we're in C++-land.

> > Querying capabilities for that type are pretty limited and quite
> > un-C++ like. Examples use a C-style switch/extractor paradigm instead
> > of a much safer variant-like visitor approach.
>
> visit() is a planned feature which is already implemented:
>
> <https://github.com/vinniefalco/json/tree/visit>

Good.

> JSON Path and JSON Pointer are planned features:
>
> <https://goessner.net/articles/JsonPath/>
>
> <https://tools.ietf.org/html/rfc6901>

Nice.

> That is by design. Boost.JSON is not a serialization library. It
> addresses the use case for a JSON DOM with accompanying parser and
> serializer. This is not an insignificant use-case; witness the
> popularity of RapidJSON and nlohmann's JSON for Modern C++. In fact
> didn't you say you are already using RapidJSON?

I use RapidJSON because it's the established industry standard to do
precisely what this library does.
As I said I don't believe the value proposition of this library is
sufficient to replace existing things when all it does is the same
thing but with a Boost logo on it. There is no guarantee at this stage
this library will get any traction or maintenance on the same level as
one which is used by thousands of projects, so it's not worth the risk
to me.