[boost] [tokenizer] fix for msvc-stlport

From: Joaquín Mª López Muñoz (joaquin_at_[hidden])
Date: 2004-09-30 12:26:44

tokenizer currently fails with tests example (regression)
and simple_example_3 and simple_example_4 (expected)
under msvc-stlport:

http://tinyurl.com/3mkvx
http://tinyurl.com/44pbx
http://tinyurl.com/5xd8z

I've tracked down the failure to a problem of MSVC 6.0 with
templatized memfuns of an imported class (thanks to Francois
Dumont):

http://www.stlport.com/dcforum/DCForumID6/1763.html

The following workaround in boost/token_functions.hpp solves the
problem for all tests. As the patch is strictly msvc-stlport-specific,
I don't think it can have nasty side effects somewhere else. OK to
commit?

Joaquín M López Muñoz
Telefónica, Investigación y Desarrollo

11a12,13
> // 30 Sep 2004   Joaquín M López Muñoz
> //      Workaround for a problem with string::assign in msvc-stlport
38a41
> #include <boost/detail/workaround.hpp>
210a214,224
> 
> #if BOOST_WORKAROUND(BOOST_MSVC, == 1200) &&\
>     BOOST_WORKAROUND(__SGI_STL_PORT, < 0x500) &&\
>     defined(_STLP_DEBUG)
>     // Problem with string::assign for msvc-stlport in debug mode: the
>     // linker is not able to import the templatized version of this memfun.
>     // See http://www.stlport.com/dcforum/DCForumID6/1763.html for details.
> 
>       t = Token();
>       while(b != e) t += *b++;
> #else
211a226,227
> #endif
>