Re: [Boost-users] <boost::wave> Memory issue when using wave to run a file parser service

Subject: Re: [Boost-users] <boost::wave> Memory issue when using wave to run a file parser service
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2014-06-12 09:29:17

> I run into a memory issue when uses boost wave library. My application
> runs as a service that parses c files. In each individual run, it calls in
> the wave library through a sub-function call to process one file. I
> noticed that some memory is not released from wave after each run. The
> consequence is that the memory usage of the application keeps increasing
> till the system breaks down.
> 
> I could reproduce the problem with a simple modification of the
> quick_start sample code shipped with the boost wave library. (attached
> below).
> 
> My system is win64 and I see the problem with both 1.49 and 1.55. (built
> with b2 toolset=msvc-11.0 address-model=64 --build-type=complete)
> 
> Something I found during the investigation:
> * The problem is gone if I build the application with link<static>.
> (Unfortunately, this is not an option for me)
> * From the memory usage pattern, it seems that there is a memory pool in
> place that should have been released after each turn.
> When repeatedly passing a 4MB file, the memory usage (taken from task
> maneger) is: 99,700K, 198,404K, 395,404K, 395408K, and 789,396K.
> When running with static link, the memory usage after each run is: 1800K,
> 2480K, 2480K, .... The maximum usage is 8300K.
> Is there any workaround with the problem?

Thanks for reporting this. I'll have a look asap.
Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu

> ----
> 
> Thanks,
> Yong
> 
> ------------------------------------- begin of the code ------------------
> ---------------------------------
> 
> #define BOOST_WAVE_SERIALIZATION        0             // enable
> serialization
> #define BOOST_WAVE_BINARY_SERIALIZATION 0             // use binary
> archives
> #define BOOST_WAVE_XML_SERIALIZATION    1             // use XML archives
> //////////////////////////////////////////////////////////////////////////
> /////
> //////////////////////////////////////////////////////////////////////////
> /////
> //  Include Wave itself
> #include <boost/wave.hpp>
> //////////////////////////////////////////////////////////////////////////
> /////
> // Include the lexer stuff
> #include <boost/wave/cpplexer/cpp_lex_token.hpp>    // token class
> #include <boost/wave/cpplexer/cpp_lex_iterator.hpp> // lexer class
> // sub-function that calls into wave
> int do_it(std::string filename) {
>     boost::wave::util::file_position_type current_position;
>     std::ifstream instream(filename);
>     std::string instring;
>     instream.unsetf(std::ios::skipws);
>     instring =
> std::string(std::istreambuf_iterator<char>(instream.rdbuf()),
>                                 std::istreambuf_iterator<char>());
>     typedef boost::wave::cpplexer::lex_token<> token_type;
>     typedef boost::wave::cpplexer::lex_iterator<token_type>
> lex_iterator_type;
>     typedef boost::wave::context<std::string::iterator, lex_iterator_type>
> context_type;
>     typedef boost::wave::cpplexer::lex_token<> TokenType;
>     typedef boost::wave::cpplexer::lex_iterator<TokenType> LexIterator;
>     LexIterator aIter(instring.begin(), instring.end(),
>     TokenType::position_type(
>         filename.c_str()),
>         boost::wave::language_support(
>             boost::wave::support_cpp
>             | boost::wave::support_option_long_long
>         )
>     );
>     LexIterator endIter;
>     for (; aIter != endIter; )  {
>         (*aIter).get_value();
>         ++aIter;
>     }
>     return 0;
> }
> int main(int argc, char *argv[])
> {
>     int aNum = 1;
>     for (;;) {
>         do_it("c:\\memleak.txt");
>         std::cout << "Process the file again (0/1)? \n";
>         scanf("%d", &aNum);
>         if (aNum == 0) {
>             printf("\nYou entered zero so we stop!\n");
>             return 0;
>         }
>     }
>     return 0;
> }