Re: [boost] [locale] Formal review of Boost.Locale library starts tomorrow

Subject: Re: [boost] [locale] Formal review of Boost.Locale library starts tomorrow
From: Artyom (artyomtnk_at_[hidden])
Date: 2011-04-15 15:33:34

> Fabio Wrote:

> 

> > Please explicitly  state in your review whether the library should be
> > accepted.
> 
> Yes.  (But I'm not quite sure if the format issue below might warrant a 
>conditional  Yes)
> 

Thanks I'm glad.

> 
> >      - What is your evaluation of the  design?
> 
> I generally like it.
> 
> I particularly like that it  integrates nicely with the std facilities (esp. 
>the use of std::string),
> and it  integrates nicely with de-facto standard tools for localization and 
>translation.  (gettext)
> 
> As far as I recall from earlier discussions it would be  possible to add a 
>backend
> to be able to use qt's translation files, too. This  would be great, because 
it
> would then be possible to use the same toolchain  (String extraction and
> Translation tools) for Application backend code when one  uses qt for the 
> Application GUI. (Even if not the QT Linguist tool is highly  sophisticated,
> so being able to use it would be a great boon)
>

There is no technical limitations to add Qt message catalogs (as they
very similar to gettext in their concepts), however
in order to do this I need a documentation of the file format.

(That it what I told last time we discussed it)

It should not be too hard but requires non-negligible amount of work.
 
> I wouldn't  require such a backend for inclusion, but if/when such a
> backend gets written  I would like it to be included.
> 
> 
> Some criticisms:
> 
> * it would be  nice if boost::locale::format would also
> understand normal boost::format  messages, or a compatible wrapper
> like boost::locale::compat::format that did  the right thing.
> I think this is important! It would be very unfortunate to  have two slightly 
>incompatible facilities.
> 

The major problem is that boost::format has too limited functionality
for localization. So I can duplicate its functionality but should I?

It is better to make users to understand the difference between two.

Another point is that boost::format throws on almost any error while
boost::locale::format ignores error (as you don't want a translator
to crash the program)

> 
> * it would be nice if  boost::locale::date_time and boost::date_time interacted 
>nicely, or ideally  could be merged
> 

The problem is that they too different.

-   boost::locale::date_time is:
  - locale dependent
  - provides non-Gregorian calendars
  - represents the time as single time point

- boost::date_time:: date and ptime has several problems.

  See: 
http://cppcms.sourceforge.net/boost_locale/html/appendix.html#why_plain_number

  So they can't be really merged. 

> * No boundary analysis in non ICU backends. it would be  nice if at least 
>simple cases
> (only single codepoint chars) would work  correctly.

Boundary analysis (or text segmentation) is an algorithm
defined by the Unicode standard. Its implementation (even
simple) requires Unicode character database and many other
things. So I don't really want to include broken or
partial algorithms especially when they quite complex
and standard API does not provide them.

I hope once Boost.Unicode will be there it would be simpler.

>[snip]
>
> One thing I miss is a more throughout documentation of the  toolchain
> needed to extract strings, translate them, ... 
> At least an example for  each major platform with all the needed
> programms. Bonus Points for build system  (bjam and CMake) integration.
> 

Good point. I'll add this to documentation.

> Some things are not quite clear
> 
> * is  there a native program to extract translation strings for each major 
>platform?  (Win-Mac-Linux)?

See: 
http://cppcms.sourceforge.net/boost_locale/html/messages_formatting.html#msg_qna

GNU Gettext tools are cross platform and available on all platforms.

Basically you will always need:

- GNU Gettext tools like xgettext msgfmt msgmerge
- Translation program like Lokalize or poedit

> 
> * how do locale::format and locale::translate interact?  will a string using 
>format get translated? can I use all of formats niceness in  translate?

See: 
http://cppcms.sourceforge.net/boost_locale/html/localized_text_formatting.html

It works this way:

   out << format(translate("Something {1}")) % param;

   - translate creates message object that holds "Something {1}"
   - format created and receives this message object 
   - a "param" binded to the format and it keeps a reference on it
   - format is written to stream upon this it:
      converts "Something {1}" to localized message like 
      "Some-Thing-in-my-language {1}" 
      format parses the localized message and renders the binded param.

Note translator can change string so if the original is "Today {1,date=short}"
translator can change it to "Today {1,date=full}" or "Today {1,ftime='%A'} 
{1,ftime='%d/%m'}"
according to his needs.

I'll add documentation of this.

> 
> * some examples on what exactly the limitations of non ICU  backends are.
> Something along the lines of if you do this on string x the result  is y 
>instead of z as it would be with ICU.

Good point. I'll try to make to be more specific.
> 
> As always when commenting on  documentation I would like more How-to and 
>walk-through examples, especially in  the reference section. (somewhat like the 
>qt docu which has a nice minimal code  sniplett for each function, which shows 
>how the function is used to achive  something)

Noted.

> Regards, and keep up the good  work
> 

Thank you for the review,
  Artyom