Re: [boost] [Boost.Locale] Questions (was New Release)]

Subject: Re: [boost] [Boost.Locale] Questions (was New Release)]
From: Artyom (artyomtnk_at_[hidden])
Date: 2010-03-23 17:07:28

> I did this and everything passed.  

Thanks, this is very good.

> Though when I run
> examples on the command line the output includes much that
> isn't readable.  Is this because the terminal doesn't
> handle the codepages correctly?

Unfortunately MS Windows has, how would I say quite limited support of
Unicode in console that is quite hard to configure.

Few points:

- Try to use UTF-8 locales for Boost explicitly
- Set in command prompt chcp 65001 -- UTF-8 codepage
- Define TrueType font and stuff would work (sometimes)

Generally the best for examples (under Windows):

1. Define UTF-8 locale globally (some examples pass locale as parameter
   to generator. if not, just change where gen("") called to 
   gen("en_US.UTF-8") or something like that.
2. Redirect output to file and see results.

Windows UTF-8 support is very poor so it is indeed problem.

> > So I would know that MSVC 2005 is fine as well. O
> <snip>
> I'm using Vista 32-bit with ICU 4.2.1
> 

Thanks.

> I've just started internationalising my project (Swish http://swish.sourceforge.net) using Boost.Locale so I
> may have many questions for you as I go along.  I hope
> this will helps Boost.Locale as much as it helps me.

Yes, indeed ;-) 
Feel free to ask, such questions are more then welcome.

> 
> Firstly, I'm trying to decide which format() to use:
> boost::format or boost::locale::format.  If I don't use
> currencies or dates and always use the global locale, are
> there any advantages to boost::locale::format?  The
> tutorial says of boost::format "It knows nothing about new
> Boost.Locale manipulators".  I don't understand what
> this means. 

There are two issues:

1. boost::format does not use locale of target stream but rather global
   locale. This is quite important for multi-locale systems
2. boost::locale::format is designed for localization purposes rather
   then safe printf replacement.

For example 

  cout << boost::locale::format("The time is {1,date=full,gmt}") % time(0).

Such things are impossible to do with format that is oriented for
number formatting. Even simple example like:

  cout << format(translate("I downloaded {1} files")) % n;

Even if you do not use, monetary and date-time formatting,
Arabic translators would may want to display Arabic numerals so then
would use flag {1,num}, thing that is impossible to do with Boost.Format.

Another point, consider following example:

 cout << format(translate("You have one file","You have {1} files",n)) % n;

If you use Boost.Format it would throw that extra parameter was used
when n==1, Boost.Locale.Format does not -- it considers missing parameters
or too much parameters as legal.

This is very important in order to prevent from translators shut
down your program because of bad translation in unexpected point.

> Which new manipulators?

 Manipulators like:
 
  cout << as::currency << 100 << as::date << as::gmt << now.

> 
> I notice that boost::locale::format has a constructor that
> takes a message as an argument and then does something
> internally that depends on the value of _translate but I
> don't understand why.  boost::format can be used like
> this:
>    boost::format(translate("Id string %1%"))
> % arg
> so what does the extra boost::locale::format constructor
> allow that this doesn't?
> 

For example:

  std::locale::global(gen("he_IL.UTF-8"));
  std::cout.imbue(gen("ar_EG.UTF-8"));

  std::cout << boost::format(translate("Pease")) ;
  // Outputs Shalom (Hebrew) according to **global** locale
  std::cout << boost::locale::format(translate("Hello"));
  // Outputs Salam  (Arabic) according to cout's locale

Generally boost::locale::format does not evaluate its parameters
until it is written to stream and then it uses target stream locale.

> Also I make gettext enforce correct translation of format
> strings by passing --boost to xgettext.  This reports
> an error if the format tokens, %1% etc., don't match in the
> original and translated strings.  Changing these to {1}
> etc. in boost::locale::format means this no longer
> works.  What was the technical motivation for this
> change?
> 

As I mentioned before, using {} allow you to provide much wider and
readable information that is impossible to pass with boost.format like

format(translate("Today is {1,date}")) % now ;

And translator may provide something like

format(translate("Hayom {1,date} ({1,locale=he_IL_at_calendar=hebrew,date})")) % now ;

-- Change locale temporary and provide date in two calendars
Hebrew and international (Gregorian).

Such flexibility require very flexible parametrization of formatting.

Also please note, these warning are mostly directed to type-unsafe
printf -- in order to prevent from translators crashing the program
by passing bad parameters. boost::locale::format is type-safe and
tolerant to invalid flags.


> My second question relates to the directory structure for
> the .mo files and applies to gettext as much as to
> boost::locale (I'm new to gettext too).  Why do they
> expect to find the files at
> lang_ctry/LC_MESSAGES/proj.mo?  This makes sense when
> translations for many projects are stored in a common
> location such as |/usr/share/locale/ but on Windows this
> wouldn't be likely.  I plan to store my translation in
> my program's installation directory and wish I could just
> put them in a flat directory: translations/ar.mo
> translations/en_GB.mo etc.  Is it possible to make the
> lookup fallback to this scheme or even make the scheme
> customizable?
> 

I follow the gettext standards, even Windows programs that use
gettext (like gimp) follow it and place it under:

c:\program files\gimp\locale\en\LC_MESSAGES\gimp.mo

Also consider possibility that you may have many modules each one can
be translated in different domain, so the different domains would
not collide like:

  c:\program files\Foo\foo.exe
  c:\program files\Foo\bar.dll
  c:\program files\Foo\locale\en\LC_MESSAGES\foo.mo
  c:\program files\Foo\locale\en\LC_MESSAGES\bar.mo
  c:\program files\Foo\locale\es\LC_MESSAGES\foo.mo
  c:\program files\Foo\locale\es\LC_MESSAGES\bar.mo

It is impossible to do this with single directory.

> Many thanks for this excellent library.
> 
> Alex
> |
> 


Thanks,
looking forward for more inputs.

Artyom