Re: [Boost-users] [Spirit] parsing double precision issues

Subject: Re: [Boost-users] [Spirit] parsing double precision issues
From: Michael Powell (mwpowellhtx_at_[hidden])
Date: 2018-11-22 16:35:02

Quite possibly this is related to an issue reported some three years
ago. How did it break, first of all? Second, why is it still broken.
Today what is clear is that it does not "just work" any longer.

https://svn.boost.org/trac10/ticket/11608
On Sat, Nov 17, 2018 at 3:38 PM Michael Powell <mwpowellhtx_at_[hidden]> wrote:
>
> On Wed, Nov 14, 2018 at 6:35 PM Michael Powell <mwpowellhtx_at_[hidden]> wrote:
> >
> > On Wed, Nov 14, 2018 at 5:05 PM Michael Powell <mwpowellhtx_at_[hidden]> wrote:
> > >
> > > On Wed, Nov 14, 2018 at 1:58 PM Michael Powell <mwpowellhtx_at_[hidden]> wrote:
> > > >
> > > > Hello,
> > > >
> > > > I am continuing to work through my Protobuf parsing with
> > > > Constant/Floating-Point tests. Double precision is rearing its ugly
> > > > head and I am curious what's the best way to handle potentially
> > > > determining a healthy epsilon given the expected parsed value. For the
> > > > most part, I think it's going to work, but for the precision issues.
> > > > Catch:
>
> No response? Does anyone have any insights as to the real parser policies?
>
> The documented grammar would lead me to believe that "fixed format",
> literally, (*digits  >> '.' >> +digits) | (+digits  >> '.' >>
> *digits), in Qi phraseology, might be possible. However, I am finding
> potentially this is not the case.
>
> > > After doing some background reading on floating point comparison ...
> > > it's not so simple as a==b, but I kind of knew that already. Even the
> > > simple absolute difference epsilon check is not so robust. I did a
> > > combination of nearly, relatively, essentially, etc, checks, and I
> > > think I have gotten past that immediate issue.
> > >
> > > Now I actually have a parsing issue... According to the Protobuf v2
> > > spec, this should parse, I think. However, it is failing to parse.
> > >
> > >   CHECK( parse_tried == expected_tried )
> > > with expansion:
> > >   false == true
> > > with message:
> > >   Source: syntax = 'proto2';option a =
> > > 933252761176835091419019014852578134454945897134240121039056292642745934626544776679309792643426912681734775655456099849022472225507009671635655261635494179939591695082743337844489696175391517474159955183289891072214770215979051082304343590361421008538352849939802258131368544983961387362613078430270357504.000000;
> > >   REQUIRE( tried )
> > > with expansion:
> > >   false
> > > with message:
> > >   Source: syntax = 'proto2';option a =
> > > 933252761176835091419019014852578134454945897134240121039056292642745934626544776679309792643426912681734775655456099849022472225507009671635655261635494179939591695082743337844489696175391517474159955183289891072214770215979051082304343590361421008538352849939802258131368544983961387362613078430270357504.000000;
> >
> > This much I can say at the moment, it does not appear to be a max
> > double issue, literally:
> >
> > 179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0000000000000000
> >
> > That would stand to reason, though, since my test data is randomly
> > generated within max double boundaries.
> >
> > I am scratching my head on the strict real policies, however. If the
> > docs are accurate, I do not see why that would not accept the fixed
> > double format...
> >
> > But it could be a concern that parsing is done apart from Qi rules. It
> > seems like some rigid logic embedded there.
> >
> > https://www.boost.org/doc/libs/1_68_0/libs/spirit/doc/html/spirit/qi/reference/numeric/real.html
> >
> > > >   REQUIRE( actual == expected )
> > > > with expansion:
> > > >   struct my::protobuf::ast::floating_point_t { 'val': 1.16624e+306,
> > > > 'opt_sign': null }
> > > >   ==
> > > >   struct my::protobuf::ast::floating_point_t { 'val': 1.16624e+306,
> > > > 'opt_sign': null }
> > > > with messages:
> > > >   Source: syntax = 'proto2';option L = 1.16624e+306;
> > > >   Delta was: 2.35309e+300
> > > >
> > > > There was clearly a delta involved, most like a precision issue
> > > > encountered during the parse. Without going into a great deal of
> > > > depth, if possible:
> > > >
> > > > namespace my { namespace protobuf { namespace ast {
> > > > using float_t = double;
> > > > enum num_sign_t {
> > > >     sign_none
> > > >     /// '-'
> > > >     , sign_minus
> > > >     /// '+'
> > > >     , sign_plus
> > > > };
> > > >
> > > > // Which includes
> > > > struct floating_point_t : numeric_t<float_t> { /* ... */ };
> > > > } } }
> > > >
> > > > // Without sign, dot required.
> > > > // >>> TODO: TBD: Precision issue? Or an actual double parsing issue? <<<
> > > > float_lit %= real_parser<ast::float_t, strict_ureal_policies<ast::float_t>>{};
> > > > floating_point_lit %= -num_lit_sign >> float_lit;
> > > >
> > > > qi::rule<It, ast::num_sign_t()> num_lit_sign;
> > > >
> > > > qi::rule<It, ast::float_t()> float_lit;
> > > > qi::rule<It, ast::floating_point_t()> floating_point_lit;
> > > >
> > > > https://developers.google.com/protocol-buffers/docs/reference/proto2-spec#floating-point-literals
> > > > https://developers.google.com/protocol-buffers/docs/reference/proto2-spec#constant
> > > > https://www.boost.org/doc/libs/1_68_0/libs/spirit/doc/html/spirit/qi/reference/numeric/real.html
> > > >
> > > > Cheers,
> > > >
> > > > Michael Powell