Re: [Boost-users] [serialization] nans and infinite floats in textbased archives...

Subject: Re: [Boost-users] [serialization] nans and infinite floats in textbased archives...
From: Robert Ramey (ramey_at_[hidden])
Date: 2009-03-08 14:06:59

I haven't worked with neither the streams library nor the floating
point utilities so I can't comment on specifics.  Sounds like you're
on the right track though.  The key point is that these issues are
addressed in the stream that the serialization library uses rather
than the serialization library itself.

Robert Ramey

François Mauger wrote:
> Hi Robert and all
>
> Thanks for this info. I downloaded the FPU stuff and follow the steps
> to use it.
> In my library, I use `boost::iostreams::filtering_i/ostream' objects
> to dump my text/XML or binary archives.
> I have to plug in it resources like
> `boost::iostreams::gzip_(de)compressor' or
> `boost::iostreams::bzip2_(de)compressor' then `std::i/ofstream' or
> `std::cin/cout' (using the `push' method).
>
> My guess is that if I use zipped filtered stream for text/XML
> archives, I should not make use of the :
>
>   locale my_locale(default_locale, new nonfinite_num_put<char>);
>
> within the final i/ofstream  as explained in the doc:
>
>   ofstream ofs ("test.txt");
>   ofs.imbue (my_locale); // <==== probably wrong
>   boost::iostreams::filtering_ostream fos;
>   fos.push (boost::iostreams::gzip_compressor ());
>   fos.push (ofs);
>   boost::archive::text_oarchive oa (fos, no_codecvt);
>  ... use oa...
>
> but rather on the `boost::iostreams::filtering_i/ostream' object:
>
>   ofstream ofs ("test.txt");
>   boost::iostreams::filtering_ostream fos;
>   fos.imbue (my_locale); // <==== probably right
>   fos.push (boost::iostreams::gzip_compressor ());
>   fos.push (ofs);
>   boost::archive::text_oarchive oa (fos, no_codecvt);
>   ... use oa...
>
> Am I right?
>
> Also does the order of `imbue' invocation on a stream is important:
> Should I use it before or after archive declaration or we don't mind?
>
>   ofstream ofs ("test.txt");
>   boost::iostreams::filtering_ostream fos;
>   fos.push (boost::iostreams::gzip_compressor ());
>   fos.push (ofs);
>   boost::archive::text_oarchive oa (fos, no_codecvt);
>   fos.imbue (my_locale);
>   ... use oa...
>
> thanks again for help
> regards
> frc
>
>> The serialization library text archives depend upon an underlying
>> stream to render output and parse input.  So what is needed
>> is a stream that can render NaNs and other such values in
>> a portable way.  A library containing such a stream has
>> passed review and been accepted into boost.  As far as
>> I can tell, it has yet to be checked into the SVN system. The
>> version reviewed can be found in the vault:
>>
>> floating_point_utilities_v3.zip
>>
>> Hopefully, this can help address your circumstances.
>>
>> Robert Ramey
>>
>> François Mauger wrote:
>>> Hi serializers, archivers, boosters and Robert,
>>>
>>> I've used boost:serialization (boost 1.34.1 under Linux ubuntu
>>> and/or SL4,, gcc 4.2) for one year. Thanks to this excellent
>>> library, our team store critical experimental data (3 TB using
>>> zipped text archives).
>>>
>>> Up to now, I met no problem with text/XML archives as all the
>>> stored double values pass the 'isnormal' check.
>>>
>>> Now my new data analysis programs needs to (de)serialize doubles
>>> even if they are NaNs or infinites (for it makes sense for me).
>>> Unfortunately, the load method for double fails while reading
>>> 'nan' (or 'inf') from the input for this is the standard behavior
>>> of the ostream with "! isnormal" doubles.
>>>
>>> After studying the source code for boost text oriented I/O archives
>>> in the hope to find a fix for my problem,
>>> I made an attempt to make boost aware of nans and +/-infs in
>>> a pragmatic way (I mean trying to save time and storage but keep
>>> the information and get rid of the stream error exception).
>>> I attached a tiny tarball to show the hack I've done.
>>>
>>> I'm not sure at all this is be a definitively good and elegant
>>> approach to handle nans within text based archives. So I'd
>>> appreciate to get some feedback and comments from Robert or any
>>> expert in
>>> this field. As I work only with Linux, gcc and 1.34.1, this approach
>>> is likely not to be portable... but maybe this could initiate
>>> some work to fix it in an official way.
>>>
>>> thanks for your attention and for all this very very
>>> useful material.
>>>
>>> regards
>>> frc
>>>
>>>> _______________________________________________
>>>> Boost-users mailing list
>>>> Boost-users_at_[hidden]
>>>> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>>
>>
>>
>> _______________________________________________
>> Boost-users mailing list
>> Boost-users_at_[hidden]
>> http://lists.boost.org/mailman/listinfo.cgi/boost-users