Boost::Regex - bug work-around...?
Hi there... I've been trying to implement some routines using the Boost library, primarily with the excellent RegEx library. Everything was working well enough in small tests, but when I tried running the routines on many patterns, my program would crash (it performs various pattern matching and search-and-replaces in multiple text files). I was able to determine that the crashes came on the calls to regex-related functions (though it tended to be a little unpredictable as to when, depending on the set of files I gave it). I tried changing my project settings (I use VC6, btw) to link with "debug" libraries instead of the standard ones (eg. "Debug Multithreaded"), and ran the program again, and everything worked perfectly. Unfortunately... I don't want to have my program running with all-debug libraries for performance and size reasons. However, is it possible to have my link-settings set to use standard libraries as before, but for the stuff that uses the regex library, to use a debug ".lib" file? Or... in the 'bjam' process, is it possible to say something to the effect of "when compiling the regex libraries, turn off compiler optimizations", so I can get a standard library that should work...? Hopefully that'll make sense... :P Thanks for your time, Mike.
My guess is your giving it the wrong iterators and it's overflowing in the rest of the memory. ---------------------------------- Peace and love, Tweety mitea@sympatico.ca - tweety_04_01@users.sourceforge.net YahooID: tweety_04_01
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Michael Starling Sent: Thursday, October 28, 2004 7:37 PM To: boost-users@lists.boost.org Subject: [Boost-users] Boost::Regex - bug work-around...?
Hi there...
I've been trying to implement some routines using the Boost library, primarily with the excellent RegEx library. Everything was working well enough in small tests, but when I tried running the routines on many patterns, my program would crash (it performs various pattern matching and search-and-replaces in multiple text files). I was able to determine that the crashes came on the calls to regex-related functions (though it tended to be a little unpredictable as to when, depending on the set of files I gave it).
I tried changing my project settings (I use VC6, btw) to link with "debug" libraries instead of the standard ones (eg. "Debug Multithreaded"), and ran the program again, and everything worked perfectly.
Unfortunately... I don't want to have my program running with all-debug libraries for performance and size reasons. However, is it possible to have my link-settings set to use standard libraries as before, but for the stuff that uses the regex library, to use a debug ".lib" file? Or... in the 'bjam' process, is it possible to say something to the effect of "when compiling the regex libraries, turn off compiler optimizations", so I can get a standard library that should work...?
Hopefully that'll make sense... :P Thanks for your time,
Mike.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
I've been trying to implement some routines using the Boost library, primarily with the excellent RegEx library. Everything was working well enough in small tests, but when I tried running the routines on many patterns, my program would crash (it performs various pattern matching and search-and-replaces in multiple text files). I was able to determine that the crashes came on the calls to regex-related functions (though it tended to be a little unpredictable as to when, depending on the set of files I gave it).
I tried changing my project settings (I use VC6, btw) to link with "debug" libraries instead of the standard ones (eg. "Debug Multithreaded"), and ran the program again, and everything worked perfectly.
Unfortunately... I don't want to have my program running with all-debug libraries for performance and size reasons. However, is it possible to have my link-settings set to use standard libraries as before, but for the stuff that uses the regex library, to use a debug ".lib" file? Or... in the 'bjam' process, is it possible to say something to the effect of "when compiling the regex libraries, turn off compiler optimizations", so I can get a standard library that should work...?
I think you really need to track down the cause of the problem before blaming it on regex or the compiler. Some possible options: You're passing a temporary to one of the regex functions and getting out-of-date invalidated iterators back. You're using some data that's uninitialised (zero initialised in bebug mode, but garbage in release mode). Also don't forget that you can put debug information in an otherwise release build, and then debug it. You can also add the regex source directly to your project (but define BOOST_REGEX_NO_LIB project-wide to suppress auto-linking), and control options from there if it comes to it. John.
I apologize, I realized after I sent it that there may have been more involved in the problem than I initially thought - I'm pretty sure the program's dying during the function calls, but as you point out, it may be just the rest of my program being lax with memory and using it all up, precipitating the problem. I was just getting frustrated and sometimes a night to sleep on it can help - I will try to double-check all the code, particularly the functions that allocate memory for the file contents (which then gets fed to RegEx...), and what values are getting passed to the library functions. If I still can't figure it out I'll try to do a more thorough trace. Despite that (which may well just be my own bad code from a long tiring day), I've been really impressed with the regex library, thank you for your work on it. Mike. John Maddock <john@johnmaddock.co.uk> wrote:
I've been trying to implement some routines using the Boost library, primarily with the excellent RegEx library. Everything was working well enough in small tests, but when I tried running the routines on many patterns, my program would crash (it performs various pattern matching and search-and-replaces in multiple text files). I was able to determine that the crashes came on the calls to regex-related functions (though it tended to be a little unpredictable as to when, depending on the set of files I gave it).
I tried changing my project settings (I use VC6, btw) to link with "debug" libraries instead of the standard ones (eg. "Debug Multithreaded"), and ran the program again, and everything worked perfectly.
Unfortunately... I don't want to have my program running with all-debug libraries for performance and size reasons. However, is it possible to have my link-settings set to use standard libraries as before, but for the stuff that uses the regex library, to use a debug ".lib" file? Or... in the 'bjam' process, is it possible to say something to the effect of "when compiling the regex libraries, turn off compiler optimizations", so I can get a standard library that should work...?
I think you really need to track down the cause of the problem before blaming it on regex or the compiler. Some possible options: You're passing a temporary to one of the regex functions and getting out-of-date invalidated iterators back. You're using some data that's uninitialised (zero initialised in bebug mode, but garbage in release mode). Also don't forget that you can put debug information in an otherwise release build, and then debug it. You can also add the regex source directly to your project (but define BOOST_REGEX_NO_LIB project-wide to suppress auto-linking), and control options from there if it comes to it. John. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
A typical scenario is to write a very small program which just uses the regex calls which you believe are causing the problem in release mode. If that problem still occurs, then you have something small and specific to report as a possible error in regex. If the problem does not occur then you know that something else you are doing is affecting your regex call in your actual program. In the latter case you can either work beckward from where you are to reduce your program to the point in which the program goes away, or work forward from your small program to see what is triggering the problem. "Michael Starling" <mstarling@rogers.com> wrote in message news:20041029131218.96837.qmail@web88108.mail.re2.yahoo.com... I apologize, I realized after I sent it that there may have been more involved in the problem than I initially thought - I'm pretty sure the program's dying during the function calls, but as you point out, it may be just the rest of my program being lax with memory and using it all up, precipitating the problem. I was just getting frustrated and sometimes a night to sleep on it can help - I will try to double-check all the code, particularly the functions that allocate memory for the file contents (which then gets fed to RegEx...), and what values are getting passed to the library functions. If I still can't figure it out I'll try to do a more thorough trace. Despite that (which may well just be my own bad code from a long tiring day), I've been really impressed with the regex library, thank you for your work on it. Mike. John Maddock <john@johnmaddock.co.uk> wrote: > I've been trying to implement some routines using the Boost library, > primarily with the excellent RegEx library. Everything was working well > enough in small tests, but when I tried running the routines on many > patterns, my program would crash (it performs various pattern matching and > search-and-replaces in multiple text files). I was able to determine that > the crashes came on the calls to regex-related functions (though it tended > to be a little unpredictable as to when, depending on the set of files I > gave it). > > I tried changing my project settings (I use VC6, btw) to link with "debug" > libraries instead of the standard ones (eg. "Debug Multithreaded"), and > ran the program again, and everyt hing worked perfectly. > > Unfortunately... I don't want to have my program r! unning with all-debug > libraries for performance and size reasons. However, is it possible to > have my link-settings set to use standard libraries as before, but for the > stuff that uses the regex library, to use a debug ".lib" file? Or... in > the 'bjam' process, is it possible to say something to the effect of "when > compiling the regex libraries, turn off compiler optimizations", so I can > get a standard library that should work...? I think you really need to track down the cause of the problem before blaming it on regex or the compiler. Some possible options: You're passing a temporary to one of the regex functions and getting out-of-date invalidated iterators back. You're using some data that's uninitialised (zero initialised in bebug mode, but garbage in release mode). Also don't forget that you can put debug information in an otherwise release build, and then debug it. You c! an also add the regex source directly to your project (but define BOOST_REGEX_NO_LIB project-wide to suppress auto-linking), and control options from there if it comes to it. John. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users ------------------------------------------------------------------------------ _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
I apologize, I realized after I sent it that there may have been more involved in the problem than I initially thought - I'm pretty sure the program's dying during the function calls, but as you point out, it may be just the rest of my program being lax with memory and using it all up, precipitating the problem. I was just getting frustrated and sometimes a night to sleep on it can help - I will try to double-check all the code, particularly the functions that allocate memory for the file contents (which then gets fed to RegEx...), and what values are getting passed to the library functions. If I still can't figure it out I'll try to do a more thorough trace.
~~~~~~~~~~~~~~~~~~~ One other thing I forgot to mention: some early VC6 versions did have an optimizer bug that effected std::string (and maybe other std lib code as well as the Boost.Regex which depends upon it), however provided you are up to date with service pack 5, then this should be a non-issue. The fix BTW was to build with /Oityb1 rather than /O2. John.
Thanks for your help with this - I didn't know about the newer service pack. I also tightened up some of the code that was allocating memory elsewhere in the program (forgot to free some... oops...), which fixed the problem; and I also installed SP6 just to be safe. Everything's running well now, thank you. Mike.
One other thing I forgot to mention: some early VC6 versions did have an optimizer bug that effected std::string (and maybe other std lib code as well as the Boost.Regex which depends upon it), however provided you are up to date with service pack 5, then this should be a non-issue. The fix BTW was to build with /Oityb1 rather than /O2.
John. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (4)
-
Edward Diener -
John Maddock -
Michael Starling -
tweety