Re: [boost] Regex allocator support

Subject: Re: [boost] Regex allocator support
From: Fred Sundvik (fred.sundvik_at_[hidden])
Date: 2009-12-03 05:43:44

Sorry for the previous message. Somehow I managed to send the message 
without a reply. A proper reply will follow soon.

"Fred Sundvik" <fred.sundvik_at_[hidden]> wrote in message 
news:14525.2590220936$1259836768_at_news.gmane.org...
>>> We recently had to use the regular expression library in conditions
>>> where there was no available physical memory, for reporting resource
>>> consumption when our console game runs out of memory. This wasn't
>>> possible however due to the fact that the basic_regex does some
> dynamic
>>> allocations and doesn't provide any allocator specialization,
> allthough
>>> match_results for example does. So we modified the boost code a
> little
>>> bit and added a new template paramter for the allocator, and made
>>> constructors that takes allocators as parameters.
>>>
>> <snip>
>>>
>>> So my question is, what are the chances that theese kinds of
>>> modificiations gets implemented in the actual library, and possibly
> in
>>> the next c++ standard library as well? I consider it a quite big
> design
>>> flaw to not have any control of the internal allocations, so for me
> this
>>> a quite big thing.
>>
>> +1; I agree with the OP's sentiments, though I wouldn't say it's
>> necessarily a "big" design flaw.  However, I've never used the
>> Boost.Regex library, and I don't know if the next standard specifies
>> allocator parameters.  I gather it doesn't?
>
> Sigh... no.
>
> The original regex++ library upon which the Boost version is derived
> *did*
> have allocators for everything.  But during review folks felt quite
> strongly
> that:
>
> a) It was overkill.
> b) Regex should be free to manage it's own memory - it's not a container
> -
> it's needs are much more complex than that - so it should be free to
> optimize memory allocation and caching as it sees fit.
>
> As a result this facility was removed.  Rightly or wrongly, no one on
> the
> stds committee questioned that design decision.
>
> Some specific comments on the other issues:
>
>>The mem_block_cache system was a bit more complicated. Initially we
>>disabled
>>it completely, but that obviously wasn't the best solution. So we
> figured
>>out, that we can use a custom allocator as default, instead of
>>std::allocator. That custom allocator then had to be specialized for
> the
>>mem_block_nodes, and the funcion calls to get_mem_block replaced by
>>allocator calls. Our implementation isn't ideal, though as we used
>>mem_block_node as the specialization, and to pass the correct size to
> the
>>allocator, we allocate  BOOST_REGEX_BLOCKSIZE / sizeof(mem_block_node)
> of
>>them, this way it works for allocators that don't specialize for
>>mem_block_node(but only if the size is dividable by
>>sizeof(mem_block_node) ). Better would probably be to make a new type
> that
>>is BOOST_REGEX_BLOCKSIZE big.
>
> Can you not just define BOOST_REGEX_RECURSIVE and use the stack based
> implementation that does away with that altogether?
>
>>We also found another problem with allocation, the match_result::format
>>functions only takes string_type, that isn't specialized by any
> allocator
>>as
>>they input. This made it impossible to call them with strings using
>>different allocators. So we added specializations of those functions
> for
>>different allocators and traits. There are probably more functions like
>>that, but since our code doesn't use them there was no need to change
>>anything.
>
> The current Trunk allows any string, container or character type, plus
> function objects to be used as arguments to regex_format/regex_replace
> etc,
> which should avoid that problem.
>
>>We recently had to use the regular expression library in conditions
> where
>>there was no available physical memory, for reporting resource
> consumption
>>when our console game runs out of memory.
>
> I'm not sure I understand the issue here, that operator new doesn't
> report
> out of memory conditions?  What does your allocator do that's different
> to
> operator new, and why not replace global new and delete with calls to
> your
> custom allocator?  Just trying to get a handle on the issue...
>
>>So my question is, what are the chances that these kinds of
> modifications
>>gets implemented in the actual library, and possibly in the next c++
>>standard library as well?
>
> As noted above, I was asked to remove this feature during review, so I'm
> not
> *that* keen to put it back in!  As far as the standard is concerned it's
>
> basically a done deal at this stage in the process - too late for such a
> big
> change.  WRT Boost I'd really like to get a handle on the issue better
> before making judgment - you seem to be one of very few people actually
> using custom allocators ;-)
>
> Regards, John.
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>
> _______________________________________________
> Unsubscribe & other changes: 
> http://lists.boost.org/mailman/listinfo.cgi/boost
>