Re: [Boost-users] Possible bug: regex, lexical_cast

Subject: Re: [Boost-users] Possible bug: regex, lexical_cast
From: John.X.Foster_at_[hidden]
Date: 2010-04-12 23:37:26

Apologies in advance for the long disclaimer at the end of this message!

> Note that your code does not use ostringstream, so your analysis is 
> wrong. I'm 99% sure that Steven's guess is correct. Try to recall 
> how you built boost regex library. How did you pass _SECURE_SCL 0 
> and _HAS_ITERATOR_DEBUGGING 0 to it?
> 
> Roman Perepelitsa._______________________________________________

Roman
While you are 99% sure its a boost library build problem, I am fairly 
certain it is not. The libraries were built with:

.\bjam -a --with-regex --with-thread --with-date_time --with-filesystem 
--with-program_options define=_SECURE_SCL=0 
define=_HAS_ITERATOR_DEBUGGING=0 define=_SCL_SECURE_NO_WARNINGS 
define=_CRT_NONSTDC_NO_DEPRECATE define=_CRT_SECURE_NO_DEPRECATE 
define=_SCL_SECURE_NO_DEPRECATE -d+2 > bjamout.txt

Example build output is:

file 
bin.v2\libs\regex\build\msvc-9.0\debug\link-static\threading-multi\c_regex_traits.obj.rsp
"libs\regex\src\..\src\c_regex_traits.cpp"
 
-Fo"bin.v2\libs\regex\build\msvc-9.0\debug\link-static\threading-multi\c_regex_traits.obj"
    -TP
 /Z7
 /Od
 /Ob0
 /W3
 /GR
 /MDd
 /Zc:forScope
 /Zc:wchar_t
 /wd4675
 /EHs
 -c
 
-DBOOST_ALL_NO_LIB=1
 
-D_CRT_NONSTDC_NO_DEPRECATE
 
-D_CRT_SECURE_NO_DEPRECATE
 
-D_HAS_ITERATOR_DEBUGGING=0
 
-D_SCL_SECURE_NO_DEPRECATE
 
-D_SCL_SECURE_NO_WARNINGS
 
-D_SECURE_SCL=0
 
"-I."

compile-c-c++ 
bin.v2\libs\regex\build\msvc-9.0\debug\link-static\threading-multi\c_regex_traits.obj

    call "c:\Program Files\Microsoft Visual Studio 9.0\VC\vcvarsall.bat" 
x86 >nul
cl /Zm800 -nologo 
@"bin.v2\libs\regex\build\msvc-9.0\debug\link-static\threading-multi\c_regex_traits.obj.rsp" 

You are right, though, the posted code does not use ostringstream - sorry 
for the confusion. ostringstream was used in one of the various 
distillations of code in trying to locate this error. Code similar to this 
is used in many places, all successfully, apart from two. One used 
ostringstream.

For example, replace the string concatentation:

        std::string x = "x" + boost::lexical_cast<std::string>(i);

 with:

        std::ostringstream oss;
        oss << "x" << i;
        std::string x = oss.str();

and the bug remains. Replace it with:

        std::string x = "x";
        x += (std::string)boost::lexical_cast<std::string>(i);

and it works perfectly. Also, using boost::format instead of ostringstream 
also works fine. ostringstream was also responsible for another bug of a 
similar nature - all occurring in the basic_string destructor - code 
below.

I think, therefore, that the boost libraries are probably not the problem 
(although in these cases both regex and lexical_cast are used); my 
suggestion is that the string concatentation and ostringstream use some 
common code wherein lies the possible bug.

For completeness, code was built using debug multi-threaded dll libraries: 
/MDd. Under VC++ 2005 all is fine; under 2008 the problem occurs. Also, 
code is fine in release mode. Here is another code distillation showing a 
similar problem:

#define _SECURE_SCL 0
#define _HAS_ITERATOR_DEBUGGING 0

#include <boost/format.hpp>
#include <boost/regex.hpp>
#include <boost/lexical_cast.hpp>

#include <iostream>
#include <sstream>
#include <string>

struct Version
{
        Version();
        ~Version();

        static const Version& currentVersion();

private:

        static const Version Version::currentVersion_;

        std::string version_;
        int major_;
        int minor_;
};

namespace
{

        std::string version4d()
        {
                int zero = 0;
                int one = 1;

                //boost::format fmt("%1%.%2%");
                //fmt % one % zero;
                //return fmt.str();
                std::ostringstream oss;
                oss << one << "." << zero;
                return oss.str();
        }
        const boost::regex reg("^([0-9]+)\\.([0-9]+)$");

        int group(const std::string& ver, int i)
        {
                boost::match_results<std::string::const_iterator> match;
                bool found = boost::regex_search(ver, match, reg);
                return found ? boost::lexical_cast<int>(match.str(i + 1)) 
: 0;
        }
        int major(const std::string& ver)
        {
                return group(ver, 0);
        }
        int minor(const std::string& ver)
        {
                return group(ver, 1);
        }
}
const Version Version::currentVersion_;

Version::Version():
        version_(version4d()),
        major_(major(version_)),
        minor_(minor(version_))
{
}
Version::~Version()
{
}
const Version& Version::currentVersion()
{
        return currentVersion_;
}
int main()
{
        std::cout << "Enter to exit\n";
        std::cin.get();
}

In this case, the error occurs in the destructor of the static member, 
currentVersion_. If you build and run these examples, you should be able 
to reproduce the bug. If, on the other hand, I have made some foolish 
mistake, I would be pleased to be enlightened.

John
John Foster

Senior Quantitative Credit Developer
Model Development, Group Credit Portfolio & Models
National Australia Bank

Level 15, 500 Bourke St, Melbourne VIC 3000
Tel: [phone] 
Email:  John.X.Foster_at_[hidden] 

This e-mail is sent by or on behalf of the named sender identified above.
If:

(a) you do not wish to receive any e-mail marketing material from this
person in the future, please forward the contents of this email to
unsubscribe_at_[hidden] with the word "unsubscribe" in the subject box.

(b) you wish to unsubscribe from all central e-mail marketing lists used
by our business, please forward the contents of this e-mail to
unsubscribeall_at_[hidden] with the message "unsubscribe from all central
e-mail marketing lists" in the subject box.

If you do not forward the contents of this e-mail with your
unsubscription then it may not be able to be implemented.

The information contained in this e-mail communication may be
confidential. You should only read, disclose, re-transmit, copy,
distribute, act in reliance on or commercialise the information if you
are authorised to do so. If you are not the intended recipient of this
e-mail communication, please immediately notify us by e-mail to
postmaster_at_[hidden], or reply by e-mail direct to the sender and then
destroy any electronic and paper copy of this message. Any views
expressed in this e-mail communication are those of the individual
sender, except where the sender specifically states them to be the views
of a member of the National Australia Bank Group of companies. Any
advice contained in this e-mail has been prepared without taking into
account your objectives, financial situation or needs. Before acting on
any advice in this e-mail, National Australia Bank Limited recommends
that you consider whether it is appropriate for your circumstances. If
this e-mail contains reference to any financial products, the National
recommends you consider the Product Disclosure Statement (PDS) or other
disclosure document before making any decisions regarding any products.
The National Australia Bank Group of companies does not represent,
warrant or guarantee that the integrity of this communication has been
maintained nor that the communication is free of errors, virus or
interference.