[Boost-users] problem in multithreaded regex_replace on linux

From: Ashish Varman (ashish_at_[hidden])
Date: 2006-06-19 10:00:21

Hello all

I am getting segmentation faults when we use our program in 
multithreaded mode on linux (RedHat 9).  The same code runs flawlessly 
on windows.  The crash always happens in boost::regex_replace.  I had 
read the following posts on this topic:
http://lists.boost.org/MailArchives/boost/msg59110.php
and
http://lists.boost.org/boost-users/2005/01/9712.php
Accordingly, I've initialised all the regexes before the main working 
threads starts.

I also came across the gcc string threading issue at
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21334
and switched to STLport but to no avail.  If anything, the crash happens 
much more quickly.

Stack trace is as follows:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1129827392 (LWP 3124)]
0x4013f843 in 
boost::re_detail::basic_regex_formatter<boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > >, 
boost::match_results<wchar_t const*, 
stlp_std::allocator<boost::sub_match<wchar_t const*> > >, 
boost::regex_traits_wrapper<boost::regex_traits<wchar_t, 
boost::cpp_regex_traits<wchar_t> > > >::format_all() (
    this=0x4357bcfc) at regex_format.hpp:133
133regex_format.hpp: No such file or directory.
in regex_format.hpp
(gdb) bt
Thread 3 (Thread 1129827392 (LWP 3124)):
#0  0x4013f843 in 
boost::re_detail::basic_regex_formatter<boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > >, 
boost::match_results<wchar_t const*, 
stlp_std::allocator<boost::sub_match<wchar_t const*> > >, 
boost::regex_traits_wrapper<boost::regex_traits<wchar_t, 
boost::cpp_regex_traits<wchar_t> > > >::format_all()
    (this=0x4357bcfc) at regex_format.hpp:133
#1  0x4013f72c in 
boost::re_detail::basic_regex_formatter<boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > >, 
boost::match_results<wchar_t const*, 
stlp_std::allocator<boost::sub_match<wchar_t const*> > >, 
boost::regex_traits_wrapper<boost::regex_traits<wchar_t, 
boost::cpp_regex_traits<wchar_t> > > >::format(wchar_t const*, wchar_t 
const*, boost::regex_constants::_match_flags) (
    this=0x4357bcfc, p1=0x4357bdac, p2=0x4357bdc4, f=50331648)
    at regex_format.hpp:123
#2  0x4013f5bd in 
boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > > 
boost::re_detail::regex_format_imp<boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > >, 
wchar_t const*, stlp_std::allocator<boost::sub_match<wchar_t const*> >, 
wchar_t, boost::regex_traits_wrapper<boost::regex_traits<wchar_t, 
boost::cpp_regex_traits<wchar_t> > > 
 >(boost::regex_traits_wrapper<boost::regex_traits<wchar_t, bo
ost::cpp_regex_traits<wchar_t> > >, boost::match_results<wchar_t const*, 
stlp_std::allocator<boost::sub_match<wchar_t const*> > > const&, wchar_t 
const*, wchar_t, boost::regex_constants::_match_flags, 
boost::regex_traits_wrapper<boost::regex_traits<wchar_t, 
boost::cpp_regex_traits<wchar_t> > > const&) (out=
      
{<iterator<stlp_std::output_iterator_tag,wchar_t,int,wchar_t*,wchar_t&>> 
= {<No data fields>}, out = 0x4357becc}, m=@0x80c3d70, p1=0x4357bdac,
    p2=0x4357bdc4, flags=50331648, t=@0x8077d48) at regex_format.hpp:569
#3  0x4013f28c in 
format<boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > >, 
boost::basic_regex<wchar_t, boost::regex_traits<wchar_t, 
boost::cpp_regex_traits<wchar_t> > > > (this=0x80c3d70, out=
      
{<iterator<stlp_std::output_iterator_tag,wchar_t,int,wchar_t*,wchar_t&>> 
= {<No data fields>}, out = 0x4357becc}, fmt=@0x4357bdac, flags=50331648,
    re=@0x810eefc) at match_results.hpp:170
#4  0x4013ed6b in 
boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > > 
boost::regex_replace<boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > >, 
wchar_t const*, boost::regex_traits<wchar_t, 
boost::cpp_regex_traits<wchar_t> >, 
wchar_t>(boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > >, 
wchar_t const*, 
boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > >, 
boost::basic_regex<wchar_t, boos
t::regex_traits<wchar_t, boost::cpp_regex_traits<wchar_t> > > const&, 
boost::re_detail::string_out_iterator<stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > > const*, 
boost::regex_constants::_match_flags) (out=
      
{<iterator<stlp_std::output_iterator_tag,wchar_t,int,wchar_t*,wchar_t&>> 
= {<No data fields>}, out = 0x4357becc}, first=0x4357bfdc, last=0x4357bff8,
    e=@0x810eefc, fmt=0x4357bf3c, flags=50331648) at regex_replace.hpp:53
#5  0x4013e123 in stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> > 
boost::regex_replace<boost::regex_traits<wchar_t, 
boost::cpp_regex_traits<wchar_t> >, 
wchar_t>(stlp_std::basic_string<wchar_t, stlp_std::char_traits<wchar_t>, 
stlp_std::allocator<wchar_t> > const&, boost::basic_regex<wchar_t, 
boost::regex_traits<wchar_t, boost::cpp_regex_traits<wchar_t> > > 
const&, stlp_std::basic_string<wchar_t, stlp_std::char_traits<wchar_t>, 
stlp_std::allocator<wchar_t> > const, 
boost::regex_constants::_match_flags) (
    s=@0x4357bfdc, e=@0x810eefc, fmt=@0x4357bf3c, flags=50331648)
    at regex_replace.hpp:96
#6  0x4013d9c5 in 
RegExRuleSet::ApplyRules(stlp_std::basic_string<wchar_t, 
stlp_std::char_traits<wchar_t>, stlp_std::allocator<wchar_t> >&) 
(this=0x807d93c,
    zrTok=@0x4357c12c) at common/utils/RuleApplier.cpp:192
#7  0x4014d42a in 
Transform1::PhonetizeAndSyllabify(stlp_std::vector<Item, 
stlp_std::allocator<Item> >&) (this=0x807d8e8, zrItems=@0x4188d448)
    at textProcessor/Transforms/Transform1/Transform1.cpp:620
#8  0x4014c0f9 in Transform1::Process(stlp_std::vector<Item, 
stlp_std::allocator<Item
 > >&) (this=0x807d8e8, zrItems=@0x4188d448)
    at textProcessor/Transforms/Transform1/Transform1.cpp:326
#9  0x4012f327 in Fragment::ProcessTransforms(int) (this=0x4188d448,
    zLangIndex=0) at common/shared/Fragment.cpp:187
#10 0x40119ef8 in Engine::ProcessFragment() (this=0x4357c50c) at 
main/Engine.cpp:64
#11 0x401110b4 in Input2Stage1(wchar_t const*, Engine&, int) (
    zpText=0x80b9cb0, zrEngineObj=@0x4357c50c, zLangIndx=0)
    at main/InterfaceFunctions.cpp:137
#12 0x401039da in GetOutput (zpText=0x80b9cb0, zSamplingRate=16000, 
zBitRate=16,
    zTimeModFact=1, zPitchModFact=0,
    zpOutFileName=0x4357c60c "./buffer/Thread_1_Line_3.wav", zLangIndx=0,
    zOutput=WAVEFILE, zppWaveBuffer=0x0) at main/Interface.cpp:185
#13 0x08056ea4 in GetOutputer(void*) (zpThreadPara=0xbfffe5b0)
    at testing/MultiThreadedApp.cpp:194
#14 0x404da2b6 in start_thread () from /lib/tls/libpthread.so.0

The code fragment is as follows

    vector <Rule>::iterator yIterInRule = this->inclusiveRuleSet.begin();
    for (; yIterInRule != inclusiveRuleSet.end(); yIterInRule++) {
        const boost::wregex &ySearchExp = yIterInRule->getSearchExpr();
        yReplaceStr.assign(yIterInRule->getReplaceExpr());
        if (yIterInRule->getContinousStatus() == false) {
            yInputStr = regex_replace(yInputStr,ySearchExp,
                          yReplaceStr,               
                          boost::format_all |
                          match_default |
                          format_sed);
        } else {
            do {
                yInputStr = boost::regex_replace(yInputStr,ySearchExp,
                                yReplaceStr,
                                boost::format_all |
                                match_default |
                                format_sed);
                ystart = yInputStr.begin();
                yend = yInputStr.end();
            } while (boost::regex_search(ystart, yend,ySearchExp));
        } // End Else               
       
    }// end of inclusiveRuleSet for loop

It crashes in any of the two regex_replace calls.

OS is Red Hat Linux 9
The compiler is gcc-3.2.2
STLport is STLport-5.0.2
boost is boost_1_33_1
Crash happens only on multiprocessor machines.

It would overcome a great roadblock if someone helps

Thanks in advance

Yours Sincerely
Ashish Varman