Re: [boost] Call for interest for native unicode character and string support in boost - updated definition

From: Rogier van Dalen (rogiervd_at_[hidden])
Date: 2005-07-30 10:12:24

Hi Graham,

I fear we might be heading off in a new direction here. I think
enabling third parties to add codepoints, if desirable at all (I still
have my sincere doubts) should not be a primary concern. I propose we
focus on the interface first, and leave out the implementation bits.

Discussing the implementation, I'm afraid, will just make the
discussion unclear, as I'll show by pointing out some things once.
- as for the "functions" struct, do you realise that C++ has a
built-in feature called "virtual functions" to do things like this?
- Have you realised what happens when deque<>::iterators, or any other
iterators to containers with non-contiguous elements, are fed into the
get_uppercase method with the current implementation? What about
iterators that process UTF-8 and pretend it to be a UTF-32 sequence?
- Do you realise that inline functions are not macros and thus need no
backslashes at the end of lines?
The implementation won't help discussing the interface, so I think
we'd better leave them out for now

BTW, now looking at the code: do you fully realise how iterators work?
It seems to me that
StartOfGrapheme(functions* pFns, inputIterator chPrev, inputIterator
ch, inputIterator chNext)
is not really needed, because it is quite easy to find an iterator to
the next and previous position given the current one.

Please find attached a modified version of the header. Changes are:
- It now looks like C++: superfluous semicolons are deleted and
identifiers are lowercase with underscores;
- the last "uni" prefixes have been removed;
- it does not show the implementation any more;
- I corrected some spelling errors, possibly introduced new ones
(seperable is spelled sepArable, isn't it? I'm not 100% sure);
- a get_category() function is defined to get the general General
Category (i.e. letter, mark, number, etc.)
- I deleted the page0() function because I didn't see why it should be
there (feel free to move it back in if I missed something).
- I provided only iterator-based grapheme, word and sentence skipping functions;
- I provided a locale with a few lines sketching an idea of what a
collation object should look like. Did you realise that string
comparison should likely be passed to a function or container as a
comparison object? For example, my specification would allow:

// Some container c of strings
// Some string s
std::lower_bound (c.begin(), c.end(), s,
unicode::default_locale().collate_accents());

Any comments are, of course, most welcome!

Regards,
Rogier