Re: [boost] [Spirit-general] [karma][fusion] Signed integer members of Boost.Fusion adapted ADTs are not output correctly with Boost.Spirit.Karma rules

Subject: Re: [boost] [Spirit-general] [karma][fusion] Signed integer members of Boost.Fusion adapted ADTs are not output correctly with Boost.Spirit.Karma rules
From: Torsten Maehne (Torsten.Maehne_at_[hidden])
Date: 2011-12-28 19:48:27

Hello Jeroen,

Jeroen Habraken wrote:

> Hi,
>
> On 28 December 2011 15:38, Torsten Maehne <Torsten.Maehne_at_[hidden]> wrote:
>> Hello Jeroen,
>>
>> Jeroen Habraken <vexocide_at_[hidden]> writes:
>>
>>> Hi,
>>>
>>> On 27 December 2011 19:24, Torsten Maehne <Torsten.Maehne_at_[hidden]> wrote:
>>>> Hello Jeroen,
>>>>
>>>> thank you for looking on my test case on Boost trunk! Unfortunately,
>>>> Boost trunk doesn't seem to be in a good shape currently for my test
>>>> case, as it seems to trigger additional unrelated problems. Boost 1.48.0
>>>> seems to be currently a more suitable starting point for a diagnosis of
>>>> the original problem.
>>>>
>>>> Please see my comment below regarding your suggestion.
>>>>
>>>> Jeroen Habraken wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> On 27 December 2011 00:51, Torsten Maehne <Torsten.Maehne_at_[hidden]> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I'm using Boost.Spirit.Karma to format the output of some nested structs
>>>>>> (direct access of the public member variables) and abstract data types
>>>>>> (ADTs) (access to the private member variables via getters and setters)
>>>>>> in my code. I'm using Boost.Fusion to adapt the ADTs into a sequence of
>>>>>> values of primitive data types for output with Boost.Karma. This worked
>>>>>> fine till Boost 1.44. Since Boost 1.45 up till 1.48 and the current
>>>>>> Boost trunk (I tested svn revs 75505, 75963, and 76196) the output of unsigned
>>>>>> integer members of Boost.Fusion-adapted ADTs doesn't work anymore --
>>>>>> even though I took into account the change in the name of macro from
>>>>>> BOOST_FUSION_ADAPT_CLASS to BOOST_FUSION_ADAPT_ADT as well as the
>>>>>> additionally required header
>>>>>> boost/spirit/include/support_adapt_adt_attributes.hpp.
>>>>>>
>>>>>> The problem is demonstrated in the attached test case for rational
>>>>>> number ADTs and structs, which use signed integer member
>>>>>> variables. Compilation of this test case against Boost 1.44.0 will
>>>>>> succeed and it will run without any errors. Compilation of this test
>>>>>> case against Boost 1.45.0 and 1.46.1 will fail. Compilation against
>>>>>> Boost 1.47.0 and 1.48.0 will succeed, but the output of negative values
>>>>>> of signed integer members of Boost.Fusion-adapted ADTs will be
>>>>>> wrong. The minus sign is output correctly, but it is followed by a wrong
>>>>>> value, which seems to be yielded by a cast from a signed to an unsigned
>>>>>> integer value instead of taking the absolute value of the signed integer
>>>>>> value. This has been observed on Mac OS X 10.7.2 (x86_64) with Xcode 4.2
>>>>>> using Apple's g++ 4.2.1, Apple's clang++ 3.0. Interestingly, compiling
>>>>>> the test case against Boost 1.47.0 or Boost 1.48.0 using MacPorts g++
>>>>>> 4.5.3, will yield more output checks to fail for Boost.Fusion-adapted
>>>>>> ADTs used in Karma output rules.
>>>>>>
>>>>>> Compilation against Boost from the Subversion trunk (I tested
>>>>>> revs. 75505, 75963, and 76196) does only succeed using Apple's clang++
>>>>>> 3.0 and execution will yield the same wrong output for negative values
>>>>>> of signed integer members of Boost.Fusion-adapted ADTs. Compilation
>>>>>> fails with Apple's g++ 4.2.1 and MacPorts g++ 4.5.3.
>>>>>>
>>>>>> I'm not able to locate the root cause for this problem, whether it is in
>>>>>> Boost.Fusion or Boost.Spirit.Karma. I would appreciate any help to
>>>>>> resolve this annoying issue. Until a solution is found, I'm stuck with
>>>>>> Boost 1.44.0.
>>>>>
>>>>> Several things seem to play a role, but I've had a look at as to why
>>>>> the code doesn't compile using boost-trunk and found the cause to be
>>>>> an ambiguity:
>>>>>
>>>>> boost/typeof/native.hpp:30:9: error: reference to 'enable_if' is ambiguous
>>>>> boost/utility/enable_if.hpp:36:10: error: candidates are:
>>>>> template<class Cond, class T> struct boost::enable_if
>>>>> boost/test/tree/decorator.hpp:184:23: error:                 class
>>>>> boost::unit_test::decorator::enable_if
>>>>>
>>>>> As enable_if is often used without being fully qualified this breaks
>>>>> in a major fashion. Since boost/test/tree/decorator.hpp seems to be
>>>>> quite new (added 2011-10-02 11:00:16 +0200) it might be best fixed
>>>>> there.
>>>>>
>>>>
>>>> On my side, I was not able to reproduce the "enable_if is ambiguous"
>>>> error related to Boost.unit_test that you've observed. I tried with
>>>> Apple g++ 4.2.1 and MacPorts g++ 4.5.3.
>>>
>>> Actually, you are. The logs
>>> https://svn.boost.org/trac/boost/attachment/ticket/6126/test_signed_integer_output_with_karma_boost_trunk_gcc42.log
>>> and https://svn.boost.org/trac/boost/attachment/ticket/6126/test_signed_integer_output_with_karma_boost_trunk_gcc45.log
>>> show:
>>>
>>> /opt/boost-trunk/include/boost/fusion/iterator/equal_to.hpp:77: error:
>>> expected nested-name-specifier before ¡enable_if¢
>>> /opt/boost-trunk/include/boost/fusion/iterator/equal_to.hpp:77: error:
>>> expected initializer before ¡<¢ token
>>>
>>> These occur because the code states "typename enable_if<Cond,
>>> T>::type" and the typename doesn't make sense in combination with
>>> boost::unit_test::decorator::enable_if. When you remove the typename
>>> (which is essentially what I did when debugging) you'll get the
>>> ambiguity error.
>>>
>>
>> Thanks for these explanations. Indeed this issue is distinct from my
>> original problem and maybe should be assigned an own Trac ticket.
>
> Yes, please do.
>
OK, I will do so tomorrow and send you the link to the ticket to review
the bug report.

>>>>>> Please note that I also submitted a bug report on this issue more than 6
>>>>>> weeks ago without any reaction so far:
>>>>>>
>>>>>> <https://svn.boost.org/trac/boost/ticket/6126>
>>>>
>>>> This ticket contains compile logs of my tests with the two mentioned
>>>> compilers.
>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Torsten Maehne
>>>>>
>>>>> Note: I've included boost_at_[hidden] in the discussion.
>>>>>
>>>>
>>>> I hope that someone has an idea how to proceed further to separate the
>>>> different problems. My original test case was designed to just expose
>>>> the signed integer output problem. I used Boost.unit_test to facilitate
>>>> its integration into a test suite. Now it seems to cause further side
>>>> effects... at least on your side.
>>>>
>>>>> Kind regards,
>>>>> Jeroen Habraken
>>>>>
>>>>> _______________________________________________
>>>>> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
>>>>
>>>> Best regards,
>>>>
>>>> Torsten Maehne
>>>
>>> I can't comment on
>>> https://svn.boost.org/trac/boost/attachment/ticket/6126/test_signed_integer_output_with_karma_boost148_gcc42_clang3.log,
>>> not (yet) sure what's going on there.
>>
>> I've created a new minimal test case (attached) without the automatisation provided
>> by Boost.unit_test. It just uses iostream, Boost.rational, Boost.Fusion,
>> and Boost.Karma and compiles fine against Boost trunk (rev. 76203).
>> However, the same wrong and compiler-dependent output by the Karma rule
>> for an ADT remains. Maybe this test case is more suitable for debugging.
>
> I've managed to reproduce the problem, and it's quite interesting to
> say the least:
>
> ~ Jeroen$ g++ --version
> i686-apple-darwin10-g++-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)
>
> ~ Jeroen$ g++ test_signed_integer_output_with_karma_minimal.cpp -o
> test -I library/boost-trunk/
> ~ Jeroen$ ./test
> rational_int_adt(2, -6) = -771751935/3 (expected: -1/3)
> rational_int_adt(-2, 4) = -771751935/2 (expected: -1/2)
>
> ~ Jeroen$ g++ test_signed_integer_output_with_karma_minimal.cpp -o
> test -O2 -I library/boost-trunk/
> ~ Jeroen$ ./test
> rational_int_adt(2, -6) = -1/3 (expected: -1/3)
> rational_int_adt(-2, 4) = -1/2 (expected: -1/2)
> rational_int_struct(2, -6) = 2/-6 (expected: 2/-6)
>
>
> ~ Jeroen$ g++-mp-4.4 --version
> g++-mp-4.4 (GCC) 4.4.6
>
> ~ Jeroen$ g++-mp-4.4 test_signed_integer_output_with_karma_minimal.cpp
> -o test -I Documents/Code/library/boost-trunk/
> ~ Jeroen$ ./test
> rational_int_adt(2, -6) = -1/1 (expected: -1/3)
> rational_int_adt(-2, 4) = -1/1 (expected: -1/2)
> rational_int_struct(2, -6) = 2/-6 (expected: 2/-6)
>
> ~ Jeroen$ g++-mp-4.4 test_signed_integer_output_with_karma_minimal.cpp
> -o test -O2 -I Documents/Code/library/boost-trunk/
> ~ Jeroen$ ./test
> rational_int_adt(2, -6) = -1/3 (expected: -1/3)
> rational_int_adt(-2, 4) = -1/2 (expected: -1/2)
> rational_int_struct(2, -6) = 2/-6 (expected: 2/-6)
>
>
> ~ Jeroen$ g++-mp-4.6 --version
> g++-mp-4.6 (GCC) 4.6.2
>
> ~ Jeroen$ g++-mp-4.6 test_signed_integer_output_with_karma_minimal.cpp
> -o test -I library/boost-trunk/
> ~ Jeroen$ ./test
> rational_int_adt(2, -6) = -771751935/3 (expected: -1/3)
> rational_int_adt(-2, 4) = -771751935/2 (expected: -1/2)
> rational_int_struct(2, -6) = 2/-6 (expected: 2/-6)
>
> ~ Jeroen$ g++-mp-4.6 test_signed_integer_output_with_karma_minimal.cpp
> -o test -O2 -I library/boost-trunk/
> ~ Jeroen$ ./test
> rational_int_adt(2, -6) = 0/0 (expected: -1/3)
> rational_int_adt(-2, 4) = 0/0 (expected: -1/2)
> rational_int_struct(2, -6) = 2/-6 (expected: 2/-6)
>
> Yes, that's four different outputs depending on the compiler version
> and optimization flags used... Hartmut has tried this using MSVC (not
> sure which version) and that worked just fine. I've currently broken
> my clang++ installation thus I can't try that, unfortunately.

I can help with the clang++ output:

$ clang++ --versionApple clang version 3.0 (tags/Apple/clang-211.12) (based on LLVM 3.0svn)Target: x86_64-apple-darwin11.2.0
Thread model: posix
$ clang++ -o test_signed_integer_output_with_karma_minimal \
test_signed_integer_output_with_karma_minimal.cpp \
-I/Users/maehne/Build/boost-trunk/ maehne_at_epsilon3:Boost 
$ ./test_signed_integer_output_with_karma_minimal
rational_int_adt(2, -6) = -32767/3 (expected: -1/3)
rational_int_adt(-2, 4) = -32767/2 (expected: -1/2)
rational_int_struct(2, -6) = 2/-6 (expected: 2/-6)
$ clang++ -O2 -o test_signed_integer_output_with_karma_minimal \
test_signed_integer_output_with_karma_minimal.cpp \
-I/Users/maehne/Build/boost-trunk/
$ ./test_signed_integer_output_with_karma_minimal
rational_int_adt(2, -6) = 0/0 (expected: -1/3)
rational_int_adt(-2, 4) = 0/0 (expected: -1/2)
rational_int_struct(2, -6) = 2/-6 (expected: 2/-6)

So also for clang++ the results depend on the optimization settings.

> I believe that I've found the culprit (a specific temporary) and could
> provide a patch, but that'll come tomorrow (I'll need to thoroughly
> test it, for obvious reasons). In the mean time someone should ask the
> GCC guys what's going on.

That would be great if you've found the culprit! The
compiler/optimization-dependent output also made me think of some
undefined behavior to be triggered by the C++ code, but, unfortunately,
I'm not familiar enough with the Boost.Fusion and Boost.Spirit codebase
to hunt it down. Thanks a lot for investing so much effort to analyze
and resolve the bug! If you need help to test the patch, I can try it
out on my side with my production code.

>>>
>>> Jeroen
>>>
>>> _______________________________________________
>>> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
>>
>> Best regards,
>>
>> Torsten
>>
>>
>> _______________________________________________
>> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
>
> Jeroen
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Best regards,

Torsten