Boost logo

Boost Users :

Subject: [Boost-users] <boost::wave> Memory issue when using wave to run a file parser service
From: Yong Huang (yong.huang_at_[hidden])
Date: 2014-06-11 14:04:18

Hi all,

I run into a memory issue when uses boost wave library. My application runs
as a *service *that parses c files. In each individual run, it calls in the
wave library through a sub-function call to process one file. I noticed that
some memory is not released from wave after each run. The consequence is
that the memory usage of the application keeps increasing till the system
breaks down.

I could reproduce the problem with a simple modification of the quick_start
sample code shipped with the boost wave library. (attached below).

My system is win64 and I see the problem with both 1.49 and 1.55. (built
with b2 toolset=msvc-11.0 address-model=64 --build-type=complete)

Something I found during the investigation:
* The problem is gone if I build the application with link<static>.
(Unfortunately, this is not an option for me)
* From the memory usage pattern, it seems that there is a memory pool in
place that should have been released after each turn.
When repeatedly passing a 4MB file, the memory usage (taken from task
maneger) is: 99,700K, 198,404K, 395,404K, 395408K, and 789,396K.
When running with static link, the memory usage after each run is: 1800K,
2480K, 2480K, .... The maximum usage is 8300K.

Is there any workaround with the problem?


------------------------------------- begin of the code

#define BOOST_WAVE_SERIALIZATION 0 // enable
#define BOOST_WAVE_BINARY_SERIALIZATION 0 // use binary archives
#define BOOST_WAVE_XML_SERIALIZATION 1 // use XML archives
// Include Wave itself
#include <boost/wave.hpp>
// Include the lexer stuff
#include <boost/wave/cpplexer/cpp_lex_token.hpp> // token class
#include <boost/wave/cpplexer/cpp_lex_iterator.hpp> // lexer class
// sub-function that calls into wave
int do_it(std::string filename) {
    boost::wave::util::file_position_type current_position;
    std::ifstream instream(filename);
    std::string instring;
    instring = std::string(std::istreambuf_iterator<char>(instream.rdbuf()),
    typedef boost::wave::cpplexer::lex_token<> token_type;
    typedef boost::wave::cpplexer::lex_iterator<token_type>
    typedef boost::wave::context<std::string::iterator, lex_iterator_type>
    typedef boost::wave::cpplexer::lex_token<> TokenType;
    typedef boost::wave::cpplexer::lex_iterator<TokenType> LexIterator;
    LexIterator aIter(instring.begin(), instring.end(),
            | boost::wave::support_option_long_long
    LexIterator endIter;
    for (; aIter != endIter; ) {
    return 0;
int main(int argc, char *argv[])
    int aNum = 1;
    for (;;) {
        std::cout << "Process the file again (0/1)? \n";
        scanf("%d", &aNum);
        if (aNum == 0) {
            printf("\nYou entered zero so we stop!\n");
            return 0;
    return 0;

Boost-users list run by williamkempf at, kalb at, bjorn.karlsson at, gregod at, wekempf at