[boost] Regex allocator support

Subject: [boost] Regex allocator support
From: Fred Sundvik (fred.sundvik_at_[hidden])
Date: 2009-12-02 06:27:56

Hi,

We recently had to use the regular expression library in conditions where 
there was no available physical memory, for reporting resource consumption 
when our console game runs out of memory. This wasn't possible however due 
to the fact that the basic_regex does some dynamic allocations and doesn't 
provide any allocator specialization, allthough match_results for example 
does. So we modified the boost code a little bit and added a new template 
paramter for the allocator, and made constructors that takes allocators as 
parameters.

This was quite straightforward to do, the allocators just had to be passed 
forward to the all internal classes it uses. The shared_ptr's had to be 
changed to use the three parameter constructor, that takes a a deleter and 
allocator, to prevent them from allocating.

The mem_block_cache system was a bit more complicated. Initially we disabled 
it completely, but that obviously wasn't the best solution. So we figured 
out, that we can use a custom allocator as default, instead of 
std::allocator. That custom allocator then had to be specialized for the 
mem_block_nodes, and the funcion calls to get_mem_block replaced by 
allocator calls. Our implementation isn't ideal, though as we used 
mem_block_node as the specialization, and to pass the correct size to the 
allocator, we allocate  BOOST_REGEX_BLOCKSIZE / sizeof(mem_block_node) of 
them, this way it works for allocators that don't specialize for 
mem_block_node(but only if the size is dividable by
sizeof(mem_block_node) ). Better would probably be to make a new type that 
is BOOST_REGEX_BLOCKSIZE big.

We also found another problem with allocation, the match_result::format 
functions only takes string_type, that isn't specialized by any allocator as 
they input. This made it impossible to call them with strings using 
different allocators. So we added specializations of those functions for 
different allocators and traits. There are probably more functions like 
that, but since our code doesn't use them there was no need to change 
anything.

So my question is, what are the chances that theese kinds of modificiations 
gets implemented in the actual library, and possibly in the next c++ 
standard library as well? I consider it a quite big design flaw to not have 
any control of the internal allocations, so for me this a quite big thing.
Obviously we are doing well with the modifications we made here internally, 
but there are other applications too that would benefit, many embedded 
applications for example. Would our code changes be of any use? The changes 
are only the minimal needed to make it work in our case, and there's many 
compiler and option permutations not taken care of, so obviously a lot of 
work would still be needed. Also there's no additional unit tests made.

Fred Sundvik
Bugbear Entertainment Ltd.