Re: C++Now 2021 -- Library in a Week: Boosting Boost

From: Bjorn Reese (breese_at_[hidden])
Date: 2021-05-02 15:01:50

On 2021-05-02 15:12, Jeff Garland wrote:

>     Boost.Tokenizer could do with a modernization. It generates a lot of
>     code for even simple tokenizations, which makes it unsuitable for
>     embedded devices. Some issues to consider:
> 
> 
> Is there something specific in the implementation other than using 
> templates that causes this?

The main culprint with regard to generated code size is the use of
std::string and exceptions (with exception wrapping.)

>      Â  Â * Separate the browsing API from the conversion API.
> 
> 
> Not sure I follow -- are the sub-bullets what you mean?

Yes.

Specifically, I want to be able to iterate through an input and obtain
a view of each entry without any conversions (e.g. to integer or
unescaped strings) taking place. Conversion should only happen when I
explicitly request it.

I do not mind a more user-friendly high-level API that works as the
existing API as long as I can reach for a low-level API when I need to.

> In 2021 it seems like we should turn the whole thing into range based 
> token_view that takes string_view and provides string_view of each token.

Good idea.