Re: [boost] UTF8 library - second call for informal review

From: Rogier van Dalen (rogiervd_at_[hidden])
Date: 2006-12-06 09:37:06

Hi Nemanja,

Thanks for your reply.

On 12/6/06, Nemanja Trifunovic <nemanja_trifunovic_at_[hidden]> wrote:
> 1) In fact, I am already developing [iterator adapters] in the version 2 of the library

I suppose version 2 is the one you'd like to become part of Boost,
then? I agree that the free functions are a good stepping stone for
implementing iterators.

> 2) IO - currently it is out of the scope of this library.

Why? You can do nice things such as automatically setting the right
codecvt based on the byte order mark. Works like a bliss. Again, I do
have some code lying around for this. codecvt facets also provide more
opportunity for optimisation.

> 3) Tables for data: As I replied to Hervé, my test cases showed a version with the table to run slower (with two different compilers).

I tested it on my implemenation of the codecvt facet; I imagine the
effect is greater than the effect on iterators. Also, I didn't use one
table but a couple. Then you don't need an "if" statement at all, only
a switch. You might want to try whether that makes much of a
difference.

> 4) A string type. There are way too many C++ string types out there already, and I wanted to provide a tool for making them work with UTF-8 encoding, rather than introducing yet another string class. Probably the same philosophy as Boost String Algorithms http://www.boost.org/doc/html/string_algo.html

I'm sorry, I fail to see how that argument works exactly. Can you elaborate?
Algorithms that work on different kinds of containers: use free
functions. I follow that. The alternative is to implement them as
methods of a string class. Is that the alternative you have in mind?
That was not what I meant.
A UTF-encoded string is a container with codepoints (21-bit numbers)
in an encoded form. The encoding puts some well-formedness constraints
on the underlying bitstring. This is not unlike std::stack. Why not
put the constraints in the type?

Regards,
Rogier