Re: [boost] Fw: [locale] Formal review of Boost.Locale library

Subject: Re: [boost] Fw: [locale] Formal review of Boost.Locale library
From: Artyom (artyomtnk_at_[hidden])
Date: 2011-04-14 15:20:25

> 

> Here is my review.
> I am Japanese.
> I use Microsoft Windows OS.
> 

First of all before I begin to answer I'd like to
notice a one important point:

1. Wide characters are not second class citizens in Boost.Locale
   everything is fully supported for Wide API.

2. I do use mostly UTF-8 examples as they much more portable
   but all API is mirrored to wide one.

3. You are welcome to take a look on a libs/locale/examples/w*.cpp
   files...
4. UTF-16 even has a performance advantage on widows platform as it
   is native encoding for ICU.

> Short  conclusion.
> This library is badly designed and completely useless for  Japanese
> language and Windows programmer.
> It must be rejected or drop  support of MSVC and Windows OS.
> 
> First of all, std::locale is very poorly  designed.
> It's useless for language like Japanese.
> So anything built on  top of std::locale is useless too.
> This library is no  exception.
> 

std::locale is container not the content. The content
is filled with ICU or with Win32 API calls.

> 
> Numbers, Time and Currency  formatting and parsing
> Simply replacing words is useless for Japanese.
> But  this is not a big problem.
> The real issue is...
> 
> Messages Formatting  (Translation)
> 
> This API is very badly designed.

More specific issues?

> I really disappointed  about use of hard coded char and std::string.
> It's not just output(return  type) it matters.
> We, Japanese Windows programmer, natively use  wchar_t.
> So input(function parameter) must support it too.
> 

What is wrong with wformat or with wformat(translate(""))
or wgettext?

Every message formatting API is fully wide enabled.

> Further  reading of the document reveals that auther of this library believes
> "should  use wide strings everywhere" is a  myth.
> http://cppcms.sourceforge.net/boost_locale/html/recommendations_and_myths.html
> 
> I  strongly object about that is a myth.
> 

I'm sorry but in context of cross platform programming
wide characters quite useless.

So I can't recommend using wide characters as they may be
UTF-16 or UTF-32, however for Windows only development
wide API is fine and fully supported.

> Well, it's true you don't need to  use wide strings, but using both
> char and wchar_t is not practical.
> This  library treat wchar_t like a second class citizen.

It does not, and if you feel point to specific points.

> We should use one  encoding, one type in one program.
> For a program runs on Windows OS, encoding  should be UTF-16.
> Because Windows' native encoding is UTF-16.
> Type should  be wchar_t.
> Because MSVC's wchar_t encoding is UTF-16.
> So we use wchar_t.  All native Win32 API accept wchar_t(assuming its
> encoding is UTF-16).
> ANSI  version of Win32 API is just a wrapper of native Wide version.
> It's just not  practical converting between wchar_t and char just for
> passing it to the  locale library.

And Boost.Locale uses Wide Win32 API only for this purpose.

> 
> Besides, what encoding shoud we use for char in  Windows?
> It isn't fixed under the Windows and MSVC.
> MSVC doesn't support  UTF-8 literal.
> 
> So in order to use this library, we need to store all  UTF-8 encoded
> string in a file and load it in runtime or convert it in  runtime.
> Isn't it silly we do that for using localization library?

This is the most popular and most widely used format of
the message catalogs around.

  http://cppcms.sourceforge.net/boost_locale/html/appendix.html#why_gnu_gettext

So this is the way they work.

They converted on load and then remain UTF-16 and not converted again.

> 
> So,  we need char set conversion between UTF-8 and UTF-16 in order to
> use this  library.
> Does this library support this conversion? No.

I'm sorry? Have you read the documentation? It does.

> Why it doesn't  support char set conversion between UTFs?

It does

> Why it hardcode char and  std::string?
> 

There are few very specific places is the library
where only "char *" or "std::string" is used
for rest of purposes wide API is fully provided.

> Hard coded char and std::string parameter is really really  REALLY bad design.
> 

Once again, you probably hand't look deep enough to the
documentation.

> I suspect auther of this library doesn't have any  serious experience
> in Windows programming and languages like  Japanese.
> What kind of joke it doesn't support Win32 API backend in  MSVC?

I'm sorry? Win32API backend supported by MSVC, GCC/Mingw and even Cygwin!

> 
> "All of the examples that come with Boost.Locale are designed  for
> UTF-8 and it is the default encoding used by Boost.Locale."

There are plenty wide examples in the documentation and in 
the library sources.

> Then,  don't pretend it support MSVC and Windows OS in the first place!
> Using UTF-8  in Windows is not a practical option.
> wchar_t is NOT a second class  citizen!
> 

It is not.

I really don't understand how did you get to the conclusion.

Try to be more specific.

> Sorry about my bad language.
> But this library is just  completely useless for the Japanese Windows 
>programmer.
> It must be rejected  if this library intended to support Japanese and Windows 
>OS.
> If Boost accept  this library, dropt the support of MSVC and Windows OS
> so the Windows  programmer knows it doesn't work for them.
> 
> --
> Ryou  Ezoe

I'd suggest to look to the documentation and the examples again.

Artyom