Subject: Re: [boost] Regex allocator support
From: Fred Sundvik (fred.sundvik_at_[hidden])
Date: 2009-12-03 05:43:44
Sorry for the previous message. Somehow I managed to send the message
without a reply. A proper reply will follow soon.
"Fred Sundvik" <fred.sundvik_at_[hidden]> wrote in message
>>> We recently had to use the regular expression library in conditions
>>> where there was no available physical memory, for reporting resource
>>> consumption when our console game runs out of memory. This wasn't
>>> possible however due to the fact that the basic_regex does some
>>> allocations and doesn't provide any allocator specialization,
>>> match_results for example does. So we modified the boost code a
>>> bit and added a new template paramter for the allocator, and made
>>> constructors that takes allocators as parameters.
>>> So my question is, what are the chances that theese kinds of
>>> modificiations gets implemented in the actual library, and possibly
>>> the next c++ standard library as well? I consider it a quite big
>>> flaw to not have any control of the internal allocations, so for me
>>> a quite big thing.
>> +1; I agree with the OP's sentiments, though I wouldn't say it's
>> necessarily a "big" design flaw. However, I've never used the
>> Boost.Regex library, and I don't know if the next standard specifies
>> allocator parameters. I gather it doesn't?
> Sigh... no.
> The original regex++ library upon which the Boost version is derived
> have allocators for everything. But during review folks felt quite
> a) It was overkill.
> b) Regex should be free to manage it's own memory - it's not a container
> it's needs are much more complex than that - so it should be free to
> optimize memory allocation and caching as it sees fit.
> As a result this facility was removed. Rightly or wrongly, no one on
> stds committee questioned that design decision.
> Some specific comments on the other issues:
>>The mem_block_cache system was a bit more complicated. Initially we
>>it completely, but that obviously wasn't the best solution. So we
>>out, that we can use a custom allocator as default, instead of
>>std::allocator. That custom allocator then had to be specialized for
>>mem_block_nodes, and the funcion calls to get_mem_block replaced by
>>allocator calls. Our implementation isn't ideal, though as we used
>>mem_block_node as the specialization, and to pass the correct size to
>>allocator, we allocate BOOST_REGEX_BLOCKSIZE / sizeof(mem_block_node)
>>them, this way it works for allocators that don't specialize for
>>mem_block_node(but only if the size is dividable by
>>sizeof(mem_block_node) ). Better would probably be to make a new type
>>is BOOST_REGEX_BLOCKSIZE big.
> Can you not just define BOOST_REGEX_RECURSIVE and use the stack based
> implementation that does away with that altogether?
>>We also found another problem with allocation, the match_result::format
>>functions only takes string_type, that isn't specialized by any
>>they input. This made it impossible to call them with strings using
>>different allocators. So we added specializations of those functions
>>different allocators and traits. There are probably more functions like
>>that, but since our code doesn't use them there was no need to change
> The current Trunk allows any string, container or character type, plus
> function objects to be used as arguments to regex_format/regex_replace
> which should avoid that problem.
>>We recently had to use the regular expression library in conditions
>>there was no available physical memory, for reporting resource
>>when our console game runs out of memory.
> I'm not sure I understand the issue here, that operator new doesn't
> out of memory conditions? What does your allocator do that's different
> operator new, and why not replace global new and delete with calls to
> custom allocator? Just trying to get a handle on the issue...
>>So my question is, what are the chances that these kinds of
>>gets implemented in the actual library, and possibly in the next c++
>>standard library as well?
> As noted above, I was asked to remove this feature during review, so I'm
> *that* keen to put it back in! As far as the standard is concerned it's
> basically a done deal at this stage in the process - too late for such a
> change. WRT Boost I'd really like to get a handle on the issue better
> before making judgment - you seem to be one of very few people actually
> using custom allocators ;-)
> Regards, John.
> Unsubscribe & other changes:
> Unsubscribe & other changes:
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk