Re: [boost] [locale] Review. Internationalization library?

Subject: Re: [boost] [locale] Review. Internationalization library?
From: Artyom (artyomtnk_at_[hidden])
Date: 2011-04-16 16:17:51

> From: Vicente BOTET <vicente.botet_at_[hidden]>
> > Message du 16/04/11 15:01
> > De : "Artyom" 
> > > 
> > > I have made a diagonal reading and  I find your library really interesting. 
>I 
>
> > >think I will spent quite  more time reviewing it.
> > > 
> > > I have just a first minimal  and formal remark. When I see the overloading 
>of 
>
> > >the translate  function
> > > 
> > > message boost::locale::translate (char const  *msg)
> > > message boost::locale::translate (char const *context, char  const *msg)
> > > message boost::locale::translate (char const *single,  char const *plural, 
>int 
>
> > >n)
> > > message  boost::locale::translate (char const *context, char const *single, 
>
> >  >char const *plural, int n)
> > > message boost::locale::translate  (std::string const &msg)
> > > message boost::locale::translate  (std::string const &context, std::string 

> > >const  &msg)
> > > message boost::locale::translate (std::string const  &context, std::string 

> > >const &single, std::string const  &plural, int n)
> > > message boost::locale::translate (std::string  const &single, std::string 
>const 
>
> > >&plural, int n)
> >  > 
> > > this let me think that it would be difficult to know know  what is the 
> > message,
> > > the context, the domain, whether the  message is the singular or plural 
>form.
> > 
> > This is the general  concept of how gettext used around. So I stick with its 

> >  conversion
> > extending it, same concept used in Qt. It is quite  common.
> > 
> > I agree that it is not perfect but it seems to be  ok.
> 
> I'm sure that you will reach to provide something clearer that don't  adhere 
to
> gettext (which is hidden for the user).
>

It is not really hidden user expected to use: gettext catalogs, gettext 
tools like msgfmt or xgettext, and use localization programs that support
gettext like Lokalize or poedit.
 

Also gettext fine but it has three major drawbacks that are solved
by Boost.Locale

- Support of multiple locales in same process
- More permissive runtime (BSL vs LGPL)
- Support of wide characters (wgettext)

> 
> > > The  equivalent for (w)gettext uses the c-like idiom, i.e. uses different 
> >  >names.
> > > 
> > > std::string boost::locale::gettext (char  const *id, std::locale const 
> > >&loc=std::locale())
> > >  std::string boost::locale::ngettext (char const *s, char const *p, int n, 

> > >std::locale const &loc=std::locale())
> > > std::string  boost::locale::dgettext (char const *domain, char const *id, 
> >  >std::locale const &loc=std::locale())
> > > std::string  boost::locale::dngettext (char const *domain, char const *s, 
>char 
>
> >  >const *p, int n, std::locale const &loc=std::locale())
> > >  std::string boost::locale::pgettext (char const *context, char const *id, 

> > >std::locale const &loc=std::locale())
> > > std::string  boost::locale::npgettext (char const *context, char const *s, 
>char 
>
> >  >const *p, int n, std::locale const &loc=std::locale())
> > >  std::string boost::locale::dpgettext (char const *domain, char const 
>*context, 
>
> > >char const *id, std::locale const &loc=std::locale())
> >  > std::string boost::locale::dnpgettext (char const *domain, char const 
> > >*context, char const *s, char const *p, int n, std::locale const 
> > >&loc=std::locale())
> > > 
> > > Neither is  satisfying me. 
> > 
> > 
> > These are mostly gettext compatibility  functions so users familiar
> > with gettext would find themselves with  (almost) same API.
> 
> You are not forced to provide C-like interfaces
> in a  Boost library to content some users that know
> the gettext interface. It is quite  ugly.

translate and gettext have little bit different meaning.

translate creates an object that is convertable to localized
message according to targets's (stream) locale

while gettext takes locale as parameter

   out << translate("Hello");

May be different from

   out << gettext("Hello");

As translate would use out's locale while gettext would use 
global locale.

Also I do not see something wrong with gettext basic interface.
Most users who ever used gettext are familiar with it.

> 
> > > I was wondering if the use of Boost.Parameters could  help. 
> > > 
> > 
> > I'm not really familiar with  Boost.Parameters but from quick glance it 
>requires
> > not-so trivial  template metaprogramming but what is even more important
> > I'm afraid that  things like
> > 
> > translate(context_ = "File","Open")
> 
> I had no  issue explaining this idiom to users of a 
> library. Another question is for the  implementer of the library,
> some more explanation will be needed.
>
> >  May scary off some average C++ programmers that would not understand  what
> > is this thing 
> > 
> > context_ = "File"
> > 
> >  Finally I want Boost.Locale be used by not brilliant C++ programmers as  
>well.
> > 
> > I know what hard times I have explaining about  boost::bind to average
> > C++ programmers so making things like that would  not help too much.
> 
> Well, it is your library. It is up to you to maintain  ugly interfaces like the 
>preceding ones.
> From my side the add a -1 to the  review. Fortunately there are a lot of 
>positive parts.
> 

I really like translate(context("File"),"Open") over the first.
It is much clearer from user point of view.

> > > could be written as
> > > 
> > > cout <<  format(translate("You have 1 file in the directory",
> > > plural("You  have {1} files in the directory"),files)) % files << endl;
> > > 
> > > button->setLabel(translate(context("File"),"open"));
> >  > 
> > > With this modification, the (w)gettext could all share the  same names and 

> > >there
> > > will be no need to use c-like  names.
> > > 
> > > What you think?
> > 
> > This one more  interesting. However there an important point to check:
> > 
> >  Integration with xgettext - would it be able to read this or entire new 
> >  xgettext should be developed to extract strings.
> 
> I don't understand.  Could you clarify, please?
> For me all the gettext functions should share 
> the  same name as well as all all the wgettext.
> No need to use the prefixed 'p' 'd'  ...

I explain.

Boost.Locale does not live alone. It requires other
tools, for example nice programs like poedit or Lokalize
to translate the messages.

You should also use gettext binary tools:

- xgettext - extracts the strings from sources and creates pot files
  useable for translators
- msgfmt - convert po files to mo files
- msgmerge - merge translated files with updated strings

And much more.

See: 
http://cppcms.sourceforge.net/boost_locale/html/messages_formatting.html#extracting_messages_from_code

Now xgettext tools know to  handle C++ overloading well but I'm
not sure if it would handle tranlsate(convert("File"),"Open"). When it
handles translate("File","Open") well.

So likely that means that boost_xgettext tool should be
written (and this isn't that simple).

So I'm not sure that users would be happy to work with two
tools boost_xgettext and xgettext to extract messages
from the files especially when the project
uses gettext in may parts (including non-C++ ones)

In any case I'll check this.

> 
> > I need to think about, probably it may be an  alternative.
> 
> You could also associate the translate function to the  context object 
> 
> button->setLabel(context("File").translate("open"));
> 
> This can  be extended to the domain, 
> 
> button->setLabel(domain("D").context("File").translate("open"));
> 
> but  doesn't scale well for the plural  parameter.

This way or other all these are attempts to overcome
the limitation of C++ to have named parameters...

Maybe it is something that should be solved at
language level.

I don't know. But in C++ we usually live without
named parameters...

It is a point to think about. And I don't 
currently have fast and simple solution.

Best,
  Artyom