[serialization] facets for non-finite numbers
At the suggestion of Robert Ramey's, I have put together facets for reading and writing non-finite numbers, such as infinity and not-a-number, to text streams, in a consistent and portable way. The code, with documentation and tests, is available in the Boost Vault, serialization/non_finite_num_facets.zip. If there is enough interest in the code, I will submit it for a mini-review, in order to have it added to the serialization lib. --------------------------------------------- What does the code do? The following test fails on VC++ 7.1: stringstream s; double x = numeric_limits<double>::infinity(); double y; ss << x; ss >> y; assert(x == y); The situation gets even worse when text files are required to be portable between different platforms. But the following test succeeds: stringstream s; locale old_locale; locale tmp_locale(old_locale, new extended_num_put<char>); locale new_locale(tmp_locale, new extended_num_get<char>); s.imbue(new_locale); double x = numeric_limits<double>::infinity(); double y; ss << x; ss >> y; assert(x == y); For more info, read the documentation. ------------------------------------------------------------ --Johan Råde
Johan Råde wrote:
At the suggestion of Robert Ramey's, I have put together facets for reading and writing non-finite numbers, such as infinity and not-a-number, to text streams, in a consistent and portable way.
The code, with documentation and tests, is available in the Boost Vault, serialization/non_finite_num_facets.zip.
If there is enough interest in the code, I will submit it for a mini-review, in order to have it added to the serialization lib.
---------------------------------------------
What does the code do?
The following test fails on VC++ 7.1:
stringstream s; double x = numeric_limits<double>::infinity(); double y; ss << x; ss >> y; assert(x == y);
The situation gets even worse when text files are required to be portable between different platforms.
But the following test succeeds:
stringstream s;
locale old_locale; locale tmp_locale(old_locale, new extended_num_put<char>); locale new_locale(tmp_locale, new extended_num_get<char>); s.imbue(new_locale);
double x = numeric_limits<double>::infinity(); double y; ss << x; ss >> y; assert(x == y);
For more info, read the documentation.
------------------------------------------------------------
--Johan Råde
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I have uploaded a revised version with a more robust is_nan function. The code is in the vault: serialization/non_finite_num_facets.zip. --Johan Råde
Johan Råde wrote:
At the suggestion of Robert Ramey's, I have put together facets for reading and writing non-finite numbers, such as infinity and not-a-number, to text streams, in a consistent and portable way.
If there is enough interest in the code, I will submit it for a mini-review, in order to have it added to the serialization lib.
--Johan Råde
New version uploaded to the vault, serialization/non_finite_num_facets.zip. This version has: 1. better handling of fomatting flags 2. more robust check for not-a-number 3. more tests 4. some minor issues have been fixed 5. various code improvements Now I'm not going to make any more changes for a while. Got other things to do. --Johan Råde
Johan Råde wrote:
Johan Råde wrote:
At the suggestion of Robert Ramey's, I have put together facets for reading and writing non-finite numbers, such as infinity and not-a-number, to text streams, in a consistent and portable way.
If there is enough interest in the code, I will submit it for a mini-review, in order to have it added to the serialization lib.
Actually, I would like the other codecvt facets in boost be promoted to official status. There should be a "place" in boost for these and other facets. Robert Ramey
--Johan Råde
New version uploaded to the vault, serialization/non_finite_num_facets.zip. This version has:
1. better handling of fomatting flags 2. more robust check for not-a-number 3. more tests 4. some minor issues have been fixed 5. various code improvements
Now I'm not going to make any more changes for a while. Got other things to do.
--Johan Råde
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Robert Ramey wrote:
Johan Råde wrote:
Johan Råde wrote:
At the suggestion of Robert Ramey's, I have put together facets for reading and writing non-finite numbers, such as infinity and not-a-number, to text streams, in a consistent and portable way.
If there is enough interest in the code, I will submit it for a mini-review, in order to have it added to the serialization lib.
Actually, I would like the other codecvt facets in boost be promoted to official status. There should be a "place" in boost for these and other facets.
Robert Ramey
--Johan Råde
New version uploaded to the vault, serialization/non_finite_num_facets.zip. This version has:
1. better handling of fomatting flags 2. more robust check for not-a-number 3. more tests 4. some minor issues have been fixed 5. various code improvements
Now I'm not going to make any more changes for a while. Got other things to do.
--Johan Råde
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
------------------------------------------------------------------------
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
I think there should be a Boost facets library, with the UTF-8 codecvt facet and my non-finite number facets. There is no reason why they should be buried in the serialization library. But I'm not the right person to take care of that. My area is scientific computing and GUI programming, and I know almost nothing about locales, facets and internationalization. --Johan Råde
For the record, as a punishment for raising this issue at the UK C++ BSI group meeting, I was tasked with raising the issue for the C++ Standards group. Some rambling thoughts on this issue are at http://www2.open-std.org/JTC1/SC22/WG21/docs/papers/2006/n2022.pdf Your implementation almost certainly provides a 'proof_of_concept' for this - as if it was needed. And I suspect more than a few will be well advised to use it - otherwise they may find to their cost (possibly considerable cost) that some archives turn out to be not readable because some floating-point calculation has produced an infinity or NaN. If you don't do this, it would seem prudent to ensure that every floating-point serialized is checked for finiteness before writing to the archive, and worrying about what to do it any are not! Thanks. Paul --- Paul A Bristow Prizet Farmhouse, Kendal, Cumbria UK LA8 8AB +44 1539561830 & SMS, Mobile +44 7714 330204 & SMS pbristow@hetp.u-net.com | -----Original Message----- | From: boost-bounces@lists.boost.org | [mailto:boost-bounces@lists.boost.org] On Behalf Of Johan Råde | Sent: 30 July 2006 11:02 | To: boost@lists.boost.org | Subject: Re: [boost] [serialization] facets for non-finite numbers | | Robert Ramey wrote: | > Johan Råde wrote: | >> Johan Råde wrote: | >>> At the suggestion of Robert Ramey's, I have put together | facets for | >>> reading and writing non-finite numbers, such as infinity and | >>> not-a-number, to text streams, in a consistent and portable way. | >>> | >>> If there is enough interest in the code, I will submit it for a | >>> mini-review, in order to have it added to the serialization lib. | >>> | > | > Actually, I would like the other codecvt facets in boost | be promoted | > to official status. There should be a "place" in boost | for these and | > other facets. | > | > Robert Ramey | > | > | >>> --Johan Råde | >> | >> New version uploaded to the vault, | >> serialization/non_finite_num_facets.zip. This version has: | >> | >> 1. better handling of fomatting flags | >> 2. more robust check for not-a-number | >> 3. more tests | >> 4. some minor issues have been fixed | >> 5. various code improvements | >> | >> Now I'm not going to make any more changes for a while. | >> Got other things to do. | >> | >> --Johan Råde | >> | >> _______________________________________________ | >> Unsubscribe & other changes: | >> http://lists.boost.org/mailman/listinfo.cgi/boost | > | > | > | > | > | ------------------------------------------------------------- | ----------- | > | > _______________________________________________ | > Unsubscribe & other changes: | http://lists.boost.org/mailman/listinfo.cgi/boost | | I think there should be a Boost facets library, | with the UTF-8 codecvt facet and my non-finite number facets. | There is no reason why they should be buried in the | serialization library. | | But I'm not the right person to take care of that. | My area is scientific computing and GUI programming, | and I know almost nothing about locales, facets and | internationalization. | | --Johan Råde | | | _______________________________________________ | Unsubscribe & other changes: | http://lists.boost.org/mailman/listinfo.cgi/boost |
Paul A Bristow wrote:
For the record, as a punishment for raising this issue at the UK C++ BSI group meeting, I was tasked with raising the issue for the C++ Standards group.
Some rambling thoughts on this issue are at
http://www2.open-std.org/JTC1/SC22/WG21/docs/papers/2006/n2022.pdf
Your implementation almost certainly provides a 'proof_of_concept' for this - as if it was needed.
And I suspect more than a few will be well advised to use it - otherwise they may find to their cost (possibly considerable cost) that some archives turn out to be not readable because some floating-point calculation has produced an infinity or NaN.
If you don't do this, it would seem prudent to ensure that every floating-point serialized is checked for finiteness before writing to the archive, and worrying about what to do it any are not!
Thanks.
Paul
I read your document. It seems that we have reached more or less the same conclusions about the representation of inifinites and nan in text streams. :-) Thanks for informing me that there is a C99 standard in this area. As a VC++ 7.1 user, I have never used C99. I note that: The output from the extended_num_put facet is consistent with the C99 standard, except that I do not put signs on NaNs. I will not change that, since there is no reasonable way for a C++ program to detect that sign. (And the sign has no meaning anyway, unlike infinity where a sign sometimes, but not always, does have meaning.) The extended_num_get facet should probably be modified to accept as input all the representations allowed under the C99 standard. This means adding support for 1. infinity as an alternative to inf 2. +nan and -nan 3. nan with extra info, i.e. nan(...some characters...) Thanks, Johan
| -----Original Message----- | From: boost-bounces@lists.boost.org | [mailto:boost-bounces@lists.boost.org] On Behalf Of Johan Råde | Sent: 01 August 2006 15:49 | To: boost@lists.boost.org | Subject: Re: [boost] [serialization] facets for non-finite numbers | | Paul A Bristow wrote: | > For the record, as a punishment for raising this issue at | the UK C++ BSI | > group meeting, I was tasked with raising the issue for the | C++ Standards | > group. | > | > Some rambling thoughts on this issue are at | > | > http://www2.open-std.org/JTC1/SC22/WG21/docs/papers/2006/n2022.pdf | > | > Your implementation almost certainly provides a | 'proof_of_concept' for this | > - as if it was needed. | > | > And I suspect more than a few will be well advised to use | it - otherwise | > they may find to their cost (possibly considerable cost) | that some archives | > turn out to be not readable because some floating-point | calculation has | > produced an infinity or NaN. | > | > If you don't do this, it would seem prudent to ensure that every | > floating-point serialized is checked for finiteness before | writing to the | > archive, and worrying about what to do it any are not! | > | > Thanks. | > | > Paul | > | | I read your document. It seems that we have reached more or less the | same conclusions about the representation of inifinites and | nan in text streams. :-) Not rocket science! | Thanks for informing me that there is a C99 standard in this area. | As a VC++ 7.1 user, I have never used C99. Not sure that many C users have either ;-) http://www2.open-std.org/JTC1/SC22/WG14/www/C99RationaleV5.10.pdf http://www.dinkumware.com/c99.aspx May help. | I note that: | | The output from the extended_num_put facet is consistent | with the C99 | standard, except that I do not put signs on NaNs. I will not change | that, since there is no reasonable way for a C++ program to | detect that sign. Surely C99 signbit tests the sign bit? (And is best used for testing the sign of infinity?) (And the sign has no meaning anyway, unlike infinity | where a sign sometimes, but not always, does have meaning.) Mathematically, you are absolutely correct, but I have argued, and continue to argue strongly, that there really IS a sign bit, and that it is unnecessary, indeed wasteful, to prevent users from making some entirely private use of this potentially precious sign bit, when it costs almost nothing to simply output a '-' if the NaN has the sign bit set. | The extended_num_get facet should probably be modified to accept as | input all the representations allowed under the C99 | standard. This means adding support for | 1. infinity as an alternative to inf | 2. +nan and -nan | 3. nan with extra info, i.e. nan(...some characters...) Would be good if you can spare the time. Paul --- Paul A Bristow Prizet Farmhouse, Kendal, Cumbria UK LA8 8AB +44 1539561830 & SMS, Mobile +44 7714 330204 & SMS pbristow@hetp.u-net.com
Johan Råde wrote: | I note that: | | The output from the extended_num_put facet is consistent | with the C99 | standard, except that I do not put signs on NaNs. I will not change | that, since there is no reasonable way for a C++ program to | detect that sign.
Paul A. Bristow wrote:
Surely C99 signbit tests the sign bit? (And is best used for testing the sign of infinity?)
| (And the sign has no meaning anyway, unlike infinity | where a sign sometimes, but not always, does have meaning.)
Mathematically, you are absolutely correct, but I have argued, and continue to argue strongly, that there really IS a sign bit, and that it is unnecessary, indeed wasteful, to prevent users from making some entirely private use of this potentially precious sign bit, when it costs almost nothing to simply output a '-' if the NaN has the sign bit set.
You are right, but our goals are different. You are working on a proposal to the C++ standardization committee. I writing code that should work with existing C++ compilers and std lib implementations. So I can not assume that the C99 signbit macro is present. What do you thing of Robert Ramey's idea of making it possible to configure a stream to report an error when a non-finite number is inserted or extracted? This could be done with a new stream flag and corresponding manipulators. --Johan Råde
| -----Original Message----- | From: boost-bounces@lists.boost.org | [mailto:boost-bounces@lists.boost.org] On Behalf Of Johan Råde | Sent: 02 August 2006 09:44 | To: boost@lists.boost.org | Subject: Re: [boost] [serialization] facets for non-finite numbers | | | | > Johan Råde wrote: | > | I note that: | > | | > | The output from the extended_num_put facet is consistent | > | with the C99 | > | standard, except that I do not put signs on NaNs. I | will not change | > | that, since there is no reasonable way for a C++ program to | > | detect that sign. | | Paul A. Bristow wrote: | | > Surely C99 signbit tests the sign bit? (And is best used | for testing the | > sign of infinity?) | > | > | (And the sign has no meaning anyway, unlike infinity | > | where a sign sometimes, but not always, does have meaning.) | > | > Mathematically, you are absolutely correct, but I have argued, | > and continue to argue strongly, that there really IS a sign bit, | > and that it is unnecessary, indeed wasteful, | > to prevent users from making some entirely private use | > of this potentially precious sign bit, | > when it costs almost nothing to simply output a '-' if the | NaN has the sign | > bit set. | | You are right, but our goals are different. You are working on a | proposal to the C++ standardization committee. I writing code that | should work with existing C++ compilers and std lib | implementations. So I can not assume that the C99 signbit macro is present. OK, so why not write - or get someone else to write ;-) a C++ is_negative function, and use the signbit macro if available, or make up your own bit test if not? There are a few variations with type, endianness and compiler, of course, but not TOO many? | What do you thing of Robert Ramey's idea of making it possible to | configure a stream to report an error when a non-finite number is | inserted or extracted? This could be done with a new stream flag and | corresponding manipulators. Some check would seem extremely wise, or archives may later prove unreadable :-(( But I am uncertain how best to do it. Paul --- Paul A Bristow Prizet Farmhouse, Kendal, Cumbria UK LA8 8AB +44 1539561830 & SMS, Mobile +44 7714 330204 & SMS pbristow@hetp.u-net.com
Paul A Bristow wrote:
Johan Råde wrote: | Paul A. Bristow wrote:> | | > Surely C99 signbit tests the sign bit?
| You are right, but our goals are different. You are working on a | proposal to the C++ standardization committee. I writing code that | should work with existing C++ compilers and std lib | implementations. So I can not assume that the C99 signbit macro is present.
OK, so why not write - or get someone else to write ;-) a C++ is_negative function, and use the signbit macro if available, or make up your own bit test if not?
There are a few variations with type, endianness and compiler, of course, but not TOO many?
I'm not going to do it. I don't need the sign bit in Nan (and 0), and I don't want to dig into the binary representation of floating point numbers on different platforms. If anyone needs it, let him write it. If he does, I will add it to my code. --Johan Råde
Johan Råde wrote:
Paul A Bristow wrote:
Johan Råde wrote:
Paul A. Bristow wrote:> |
Surely C99 signbit tests the sign bit?
You are right, but our goals are different. You are working on a proposal to the C++ standardization committee. I writing code that should work with existing C++ compilers and std lib implementations. So I can not assume that the C99 signbit macro is present.
OK, so why not write - or get someone else to write ;-) a C++ is_negative function, and use the signbit macro if available, or make up your own bit test if not?
There are a few variations with type, endianness and compiler, of course, but not TOO many?
I'm not going to do it. I don't need the sign bit in Nan (and 0), and I don't want to dig into the binary representation of floating point numbers on different platforms.
If anyone needs it, let him write it.
If he does,
and he writes corresponding tests, jamfiles, documentation and includes support for the variety of compilers and libraries that boost supports.
I will add it to my code.
A very good answer. The problem at hand is to define otherwise undefined behavior that currently, can result in non-sensical programs. Your solution addresses that problem. If someone wants to address some other problem, facets can be written for that and included in the "facets library". People regularly have problems with the current situtation. When someone proposes a solution, a bunch of other people chime in with their proposal to expand the problem to the point of insolubility. The task is then too large to address and the situation goes back to where it was before. This is a boost problem.
--Johan Råde
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Robert Ramey wrote:
A very good answer. The problem at hand is to define otherwise undefined behavior that currently, can result in non-sensical programs. Your solution addresses that problem. If someone wants to address some other problem, facets can be written for that and included in the "facets library". People regularly have problems with the current situtation. When someone proposes a solution, a bunch of other people chime in with their proposal to expand the problem to the point of insolubility. The task is then too large to address and the situation goes back to where it was before. This is a boost problem.
--Johan Råde
Robert, I would never have imagined that anyone would care about negative 0 and negative NaN. But it seems that no matter how obscure something is, there is someone who cares. ;-) --Johan
Johan Råde wrote:
What do you thing of Robert Ramey's idea of making it possible to configure a stream to report an error when a non-finite number is inserted or extracted? This could be done with a new stream flag and corresponding manipulators.
Actually I was envisioning that one would just add the desired facet after the stream was created. I don't see that this would require any new stream flags or manipulators. Something like // make a new locale std::locale float_locale(std::locale::global(), new codecvt_strict_float()); fostream os("filename"); os.imbue(float_locale); //... operations on os use fixed or strict floating point output Robert Ramey
Robert Ramey wrote:
Johan Råde wrote:
What do you thing of Robert Ramey's idea of making it possible to configure a stream to report an error when a non-finite number is inserted or extracted? This could be done with a new stream flag and corresponding manipulators.
Actually I was envisioning that one would just add the desired facet after the stream was created. I don't see that this would require any new stream flags or manipulators. Something like
// make a new locale std::locale float_locale(std::locale::global(), new codecvt_strict_float()); fostream os("filename"); os.imbue(float_locale);
//... operations on os use fixed or strict floating point output
Robert Ramey
Yes, in the short run, yes. That is what I've put togegther. But Paul is working on a proposal to the C++ standardization committee to get proper handling of infinity and NaN into the standard streams, without the need for special facets. P.S. These are not codecvt facets. They are num_put and num_get facets. Different base classes. Johan Råde
Paul A Bristow wrote:
For the record, as a punishment for raising this issue at the UK C++ BSI group meeting, I was tasked with raising the issue for the C++ Standards group.
Some rambling thoughts on this issue are at
http://www2.open-std.org/JTC1/SC22/WG21/docs/papers/2006/n2022.pdf
Your implementation almost certainly provides a 'proof_of_concept' for this - as if it was needed.
And I suspect more than a few will be well advised to use it - otherwise they may find to their cost (possibly considerable cost) that some archives turn out to be not readable because some floating-point calculation has produced an infinity or NaN.
If you don't do this, it would seem prudent to ensure that every floating-point serialized is checked for finiteness before writing to the archive, and worrying about what to do it any are not!
So what is needed is a "companion" facet which traps attempts to write NaNs and throws an exception. Usage of one or the other of these facets would eliminate undefined behavior. Robert Ramey
Thanks.
Paul
--- Paul A Bristow Prizet Farmhouse, Kendal, Cumbria UK LA8 8AB +44 1539561830 & SMS, Mobile +44 7714 330204 & SMS pbristow@hetp.u-net.com
-----Original Message----- From: boost-bounces@lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Johan Råde Sent: 30 July 2006 11:02 To: boost@lists.boost.org Subject: Re: [boost] [serialization] facets for non-finite numbers
Robert Ramey wrote:
Johan Råde wrote:
Johan Råde wrote:
At the suggestion of Robert Ramey's, I have put together facets for reading and writing non-finite numbers, such as infinity and not-a-number, to text streams, in a consistent and portable way.
If there is enough interest in the code, I will submit it for a mini-review, in order to have it added to the serialization lib.
Actually, I would like the other codecvt facets in boost be promoted to official status. There should be a "place" in boost for these and other facets.
Robert Ramey
--Johan Råde
New version uploaded to the vault, serialization/non_finite_num_facets.zip. This version has:
1. better handling of fomatting flags 2. more robust check for not-a-number 3. more tests 4. some minor issues have been fixed 5. various code improvements
Now I'm not going to make any more changes for a while. Got other things to do.
--Johan Råde
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
------------------------------------------------------------- -----------
_______________________________________________ Unsubscribe & other changes:
http://lists.boost.org/mailman/listinfo.cgi/boost
I think there should be a Boost facets library, with the UTF-8 codecvt facet and my non-finite number facets. There is no reason why they should be buried in the serialization library.
But I'm not the right person to take care of that. My area is scientific computing and GUI programming, and I know almost nothing about locales, facets and internationalization.
--Johan Råde
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
So what is needed is a "companion" facet which traps attempts to write NaNs and throws an exception. Usage of one or the other of these facets would eliminate undefined behavior.
Robert Ramey
I have uploaded such facets to the vault. They are called strict_num_put and strict_num_get. However, they do not throw exceptions, they set the failbit of the stream. I noticed that, at least with VC++ 7.1., if the facet throws an exception, then the stream catches the exception and sets the failbit anyways. --Johan
participants (3)
-
Johan Råde -
Paul A Bristow -
Robert Ramey