Re: [boost] [review] Review of Nowide (Unicode)

Subject: Re: [boost] [review] Review of Nowide (Unicode)
From: Groke, Paul (paul.groke_at_[hidden])
Date: 2017-06-23 14:35:13

> > They will fail if the user-provided buffer isn't big enough.
> 
> I see.
> 
> If some I'd rather expect to extend the boost.locale functionality rather
> boost.nowide - it seems to be more appropriate (since nowide functions
> finally call utf_to_utf functions from boost.locale
> library)

I think it's convenient to have the function that does the conversion in the Nowide library itself, even if it just calls come other function. That way the user doesn't have to know or care what kind of exact conversion rules Nowide uses - they can just call nowide::widen/narrow.

> Regarding iterator input/output interface.
> 
> I must say that iterator based interface isn't right for encoding conversions.
> There was a long discussion regarding it during Boost.Locale review.
> 
> The major reason is that you don't work on characters by themselves but
> rather character sequences and intermediate states.
> 
> Since all conversion API gets some kind of "char *" and virtually every string
> class around has its "c_str()" variant I don't really see it necessary to provide
> arbitrary iterators (especially since under the hood I'll need to combine them
> into chunks of char * for processing)

I'm not suggesting an iterator-based interface. I'm just suggesting two changes. 1: change

stack_wstring wstr;
wstr.convert(input);

to

stack_wstring wstr;
widen(wstr, input);

I.e. move the conversion function out of the string class. And then 2: allow arbitrary (string) types instead of stack_wstring. Not via iterators, the widen/narrow functions could (should) simply use push_back(char) or append(char const*, char const*) to "build" the output string.

> Shouldn't be a problem.  Finally all goes to console.

Won't buffering cause trouble - especially for input?