- Boost - lists.boost.org

[OT] Boost Users/Developers Down Under?
by Dean Michael Berris 19 Apr '11

19 Apr '11

Hey Everyone! Apologies for the cross post and off-topic, but I just moved to Sydney, Australia and I was wondering whether there are any Boost users/developers that are based here who'd like to get together for a geek out session somewhere in the city? I tried to look at the Boost Users Map (http://goo.gl/O1I15) and I only saw one other Boost user in the area -- maybe there are others lurking around who'd be interested in a meet-up here? :) Cheers from down under! :D -- Dean Michael Berris http://about.me/deanberris

1 0

FW: [locale] Off-list review
by steve.bush＠neosys.com 19 Apr '11

19 Apr '11

-----Original Message----- From: boost-bounces(a)lists.boost.org [mailto:boost-bounces@lists.boost.org] On Behalf Of Vicente BOTET Sent: 19 April 2011 10:33 >Hi, I have the impression that this review has already been sent at Date: Sat, 16 Apr 2011 15:40:53 +0400 . >BTW, who sent it? I did. I am sorry for wasting everybody's time. I joined the list immediately afterwards and have followed the very lively discussion intently since then. Artyom is quite resolute in the defence of his design decisions some of which brutally practical compromises - even if theoretically ugly. Things like broken standard libraries and "Broken" csv just aren't going to be fixed. This particular library has to live in the real world of existing libraries, not a vacuum, so the exquisite design purity which is so much a part of many boost libraries is less a consideration here. One of the strengths of the library seems to be that Artyom has used it in production for some years, although apparently mainly in posix/utf-8 environment and without CJK. I seriously doubt there will be any better offer for internationalisation library than this and it may trigger a better solution later on. If that is the case then this review better try to identify any unacceptable features in the library - especially those that cannot be replaced in future versions. If there are few or none then I think it should be admitted. Steve Bush

1 0

[locale] Off-list review
by Chad Nelson 19 Apr '11

19 Apr '11

This review was sent to me privately, rather than to the list. I don't think I realized that when I received it. Apologies if I had forwarded it earlier too. -- Chad Nelson Oak Circle Software, Inc. * * * Begin forwarded message: Date: Sat, 16 Apr 2011 15:40:53 +0400 From: <steve.bush(a)neosys.com> --- What is your evaluation of the design? Very good to start with since leveraging locale for i18n/l10n is a great rallying point for developers. Translate GETTEXT might be better in a separate library to increase degrees of freedom and reduce coupling. --- What is your evaluation of the implementation? The wrapper/backend strategy provides immediate benefit while leaving the door open to progressive reimplementation. Since it is a wrapper, could PARTS of it be implemented as a header only? --- What is your evaluation of the documentation? The documentation is more than good enough to start with considering this library's potential usefulness. --- What is your evaluation of the potential usefulness of the library? This library is potentially *extremely* useful because it LEVERAGES an important chunk of standard C++ that is sadly underutilised merely due to a few design and implementation flaws. This library has potential something of the order of Boost.FileSystem. This is a VERY strategic library and any shortcomings will be resolved with time. --- Did you try to use the library? With what compiler? Did you have any problems? Didn't try. --- How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I meticulously read all documentation several times. --- Are you knowledgeable about the problem domain? Yes, we have used similar technique for Exodus http://devwiki.neosys.com and will replace those with Boost.Locale if and when it becomes available. --- Do you think the library should be accepted as a Boost library? YES - It is relatively small but has a massive effect. It has a very low cost/benefit ratio since it leverages long standing classic C++ locale design/skills.

3 3

[locale] review
by Phil Endecott 19 Apr '11

19 Apr '11

Dear All, Here is a brief review of the proposed Boost.Locale library. I know little about the internationalisation issues that this library addresses, so I will focus just on the few areas where I do have some experience. The documentation is generally of good quality but could be expanded in places as noted below. I appreciate the introduction of std::locale at the beginning; having avoided std::locale for precisely the "common critical problems" mentioned, this introduction was useful. As others have said, a similar introduction to icu, perhaps even deep links to appropriate parts of its documentation, and perhaps also to Unicode docs would be useful. I found that it was not sufficiently easy to find which header each function is in, e.g. on the charset_handling.html page the name of the header is not mentioned; if I click on e.g. from_utf I get to a reference page that also doesn't name the header. Use of strings rather than symbols: there are a few places where the library uses strings such as "UTF-8" or "de_DE.UTF8" rather than symbols like utf_8 or de_DE_UTF8. This use of strings can make things run-time errors that could otherwise be compile-time errors. Mis-typing e.g. "UTF_8" or "de-DE" is too easy. Perhasp in some cases the domain of the parameter is not known at compile time, but even in those cases, it should be possible to provide symbols for the most common choices. Collation: I would be interested to know how fast these comparisons are compared to regular std::string::operator<. If I correctly understand what transform does (more docs needed), it would be interesting to then compare the performance of std::map<...,comparator> with a std::map<> storing transformed strings. Or similarly, to compare std::sort using the comparator with std::sort of a pair<transformed string, original string>. Ideally these collation functions would take character ranges of arbitrary types, however I guess that the icu API takes char* making this unreasonable. Apparently there is no option for a NULL-terminated char*, unlike the case conversion functions, right? Case conversions: I don't know what "title case" is or what "fold case" means; the docs should define this. At least linking to the definitions of the Unicode normalisation forms would also be useful; preferably, some explanation of how they should be used would be included. I can think of two recent case-conversion problems that I have had, both related to geographic names: 1. I had a list of North American names, in "UPPER CASE". I decided to convert them to "Initial Capitals" for better appearance, but later realised that this was broken for accented names in Quebec. I could imagine some sort of "case conversion is lossy" property of a locale that would detect this; does anything like that exist? (I guess not.) 2. In a similar case I found that " 's " was being wrongly capitalised, e.g. "St. John'S Street". (This was with PostgreSQL's case conversion functions.) Again it would be useful to know whether (a) does the library try to handle this sort of thing correctly, and (b) can it tell me whether the operation that I want to do is lossy or not? For example, say I have a set of place names that are a mixture of en_US and en_CA and fr_CA but I don't know which; can I perform a conversion that works because those locales all have the same rules for e.g. ' and -? (I guess not.) Character set conversion: I think it's great that Boost might finally get character set conversion - it's overdue! However, I'm not enthusiastic about this: std::string utf8_string = to_utf<char>(latin1_string,"Latin1"); 1. It should be possible for the source character set to be a compile-time constant symbol, and a template parameter e.g. conv<latin1,utf8>(str). As it is, if I want to convert just a few bytes, it will surely spend more time decoding the charset name string than actually converting. 2. As above, ideally it should take a range of arbitrary type and output to a back_insert_iterator; presumably this is an icu limitation. 3. The error handling seems to be basic - it's "all or nothing". Considering from_utf, since this is the direction where there are more things that can be unrepresentable, I might want to: - Skip the character; - Replace with '?'; - Replace with an unaccented version, or e.g. curly-quotes to plain-quotes; - Throw an exception that aborts the whole operation; - Do the conversion up to the point of difficulty and then indicate the problem. 4. There's also the case where the input is chunked, and the chunk-breaks might fall within multi-byte characters; so individual chunks are potentially malformed while the overall stream is OK. I think the best way to handle this is with a stateful functor to do the comparison; I wrote one using iconv, like this: Iconver<latin1,utf8> latin1_to_utf8; while (!end_of_input()) { output.append( latin1_to_utf8(input.get_chunk()) ); } assert(latin1_to_utf8.empty()); // detect unconverted partial chars at end of input Also: how is the performance of this conversion? If it is being done by icu, we should have a hook (i.e. specialisation, hence my suggestion of template parameters for the character sets) so that we can provide optimised versions of important conversions. Here are my answers to the "normal questions": - What is your evaluation of the design? Generally it seems like a reasonable design for an icu wrapper. If it were not an icu wrapper, no doubt some things could be done better. - What is your evaluation of the implementation? I've looked at very little of the source. I decided to look at iconv_codepage.hpp since I have some experience with iconv. I note that a union is being used in what I believe is an undefined way: union { char first; uint16_t u16; uint32_t u32; } v; v.u16 = 1; if(v.first == 1) { return "UTF-16LE"; } else { return "UTF-16BE"; } In my own iconv wrapper code I need a const_cast because the POSIX spec misses a const on the second argument to iconv(); some platforms correct this, but not all: #if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__NetBSD__) || defined(__sun__) // Previously __APPLE__ was included in this list; presumably they have // changed their headers. If you have an older system you may need to put // it back. #define ICONV_ARG2_IS_CONST #endif This doesn't seem to be done here. There seems to be some use of reinterpret_cast which I think should be a static_cast via void*: virtual std::string convert(char_type const *ubegin,char_type const *uend) { .... char const *begin = reinterpret_cast<char const *>(ubegin); "int err" should probably be "errno_t err". There seems to be a "magic constant" of 10 to allow for the difference in size between the input and the output. Some comments would help. For example, to describe the meaning of the return values and error codes from iconv(). Based on that limited look, my view is that the code probably works, but it could be improved in many ways. - What is your evaluation of the documentation? Good enough. - What is your evaluation of the potential usefulness of the library? Useful and overdue. - Did you try to use the library? With what compiler? Did you have any problems? No, I've not compiled anything. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? 3 hours reading the docs and looking at a tiny bit of the implementation. - Are you knowledgeable about the problem domain? Not beyond the areas that I have discussed above. > Please explicitly state in your review whether the library should be > accepted. It's borderline. I was going to say "yes, but please consider updating at least the charset conversion functions in the way that I proposed". But then I looked at the source and noted what I think are well-known issues like abuse of union and reinterpret_cast, which have lowered my opinion a bit. And then I've seen some other reviews that have looked at the areas I've not considered (i.e. message translation) and they have been quite critical. Based on all of that, my view has shifted to "no, but this is worthwhile functionality, so please come back when you've resolved some of these concerns". Regards, Phil.

4 9

[locale] Strange(?) result in example code
by Noah Roberts 19 Apr '11

19 Apr '11

#include <boost/locale.hpp> #include <iostream> //#include <ctime> using namespace boost::locale; int main() { generator gen; //std::locale::global(gen("snthsnth")); // no exception std::locale::global(gen("en_US.UTF-8")); double now = time(0); std::cout << "Today is " << as::date << now << std::endl << "Current time is " << as::time << now << std::endl << "The current weekday is " << as::ftime("%A") << now << std::endl; std::cin.get(); } http://cppcms.sourceforge.net/boost_locale/html/formatting_and_parsing.html I get warnings about converting from time_t to double (which I rather expected) and then get basic, double formatted output: Today is 1.3029e+009 Current time is 1.3029e+009 The current weekday is 1.3029e+009

5 14

[locale] Review
by Noah Roberts 19 Apr '11

19 Apr '11

- What is your evaluation of the design? Functions. Seems straight forward to use. - What is your evaluation of the implementation? What I used worked. I did not look over the source. - What is your evaluation of the documentation? Good except that there are at least some errors/inconsistencies (ngettext in messages_formatting.html for example is missing an argument). Also need to be explicit about which version of 3rd party programs are required (see below). - What is your evaluation of the potential usefulness of the library? I'll probably be using it for our product (assuming it plays fine with Qt, which I don't see any reason why not) so I'd say very. - Did you try to use the library? With what compiler? Did you have any problems? Yes. MSVC2010. No problems specific to the library but did have a pretty hard time figuring out which version of gettext was necessary and finding it. Simple search for gettext win32 or windows does not turn up a version compatible with this library (--keyword=translate:1,1t syntax invalid). Had to get the recent MSYS developer toolkit (mingw + msys wasn't even enough). After a couple of hours of searching it seemed to be the only option. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I wrote a simple program and played with a few of the functions. I downloaded and installed various tools that are supposed to help with translating. Basically did a short, 4 hour evaluation of whether I could propose it for use. - Are you knowledgeable about the problem domain? Not as much as I certainly should be. - Should it be in boost? I have some concerns about the fact that it relies so much on 3rd party stuff. The gettext tools are obviously moving targets and not having an easily accessible version of xgettext for Visual Studio users is not an optimal situation. Having to install MinGW and the works shouldn't be required. Various supposed binary downloads of gettext for win32 either are not or are too old. I also found that of the tools listed for working on po files, only one both worked with plural forms and worked on win32. In fact possibly only the one worked with plural forms at all. The fact that this library implements a gettext like setup is nice, but the dependency then upon gettext to provide the source extraction tools and such is unfortunate and could certainly lead to problems where there are version conflicts. Based on my experience I'd say that clearly there are some availability issues in this area for those using Visual Studio. Since gettext is someone else's problem, I don't see how boost can safely make sure the necessary programs are available to anyone using boost. Certainly some better documentation about which version of gettext is being emulated should be included (I looked through the docs and never saw anything specifying version). Once I did find the correct version of the tools (or at least a version that responded as the documentation explained) the system seemed to work out just fine. I would like to see an alternative to a complete msys install (though I personally like to have it around, others on my team find it useless). I would guess that there should be one but I seriously couldn't find it. I would also like to see boost::locale::format work like boost::format. The documentation also calls this thing a printf like utility class but it doesn't, as opposed to boost::format, accept printf format strings. On the other hand, boost::format(translate("%.4g")) % 3.14 worked just fine so I don't know what this locale specific version is providing. All that said, I think it could be a good addition. We certainly need something like it and it at least does what I think I need from such a library.

4 13

[type_traits] Plans to add C++11 type traits?
by Beman Dawes 19 Apr '11

19 Apr '11

Now that C++11 (formerly C++0x) is done, are there any plans to add the type traits to Boost that we don't already have? See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3290.pdf --Beman

4 4

[locale] Review: Tutorial
by Gevorg Voskanyan 19 Apr '11

19 Apr '11

index.html: "Correct case conversion, case folding and normalization." Please insert a link to a resource explaining the difference between case conversion and case folding. std_locales.html: In the first code snippet "std::use_facet<std::ctype>(some_locale)" is wrong. Should be: std::use_facet<std::ctype<char> >(some_locale) "imbued into an iostream" doesn't look OK, especially with 'iostream' in fixed-width font. "imbued into an I/O stream object" perhaps. Such usage of 'iostream' is present on most other pages as well. A link near the "cout.imbue(std::locale("en_US.UTF-8")); ..." example to the "Locale names are not standardized. ..." paragraph would help. Would be good to mention std::has_facet. What is paper_size::inches? I guess it should be measure::inches. The comment below needs to be corrected too. class measure is missing the static std::locale::id id; member print_distance example should use the 'out' object instead of std::cout. "// Set default locale as global" This comment is misleading. The default locale in C++ programs is the 'classic' locale. std::locale("") is the user's preferred locale. Or perhaps we can use the system's default locale term, but I think "user's preferred" is better. formatting_and_parsing.html: Strictly speaking, "the iostream STL library" is not correct formulation. Should be e.g. "the standard I/O streams". why "double now=time(0);" and not "time_t now=time(0);" ? s/tommorrow/tomorrow/ s/display in local time format/display in local time zone/ s/as::locale_time/as::local_time/ s/Locale time is/Local time is/ conversions.html: Add code example calling to_lower and fold_case on "grüßen" to help users understand the difference between the two. messages_formatting.html: s/Is change to/Is changed to/ Is there a default search path for dictionaries (/usr/share/locale under *nix), or should an application always call add_messages_path() ? "The source text is not copied." Sometimes 'not' is highlighted, and sometimes 'is'. So what does it try to say after all? That the provided string argument must not die before the translated string is fetched? s/tanslate/translate/ s/Japaneese/Japanese/ ja << msg missing semicolon s/main()/int main()/ std::wstring msg = translate("Do you want to open the file?").str<wchar_t>(some_locale) missing semicolon s/GNU Gettext catalogs has/GNU Gettext catalogs have/ "is \c n 1 or not" looks like a doxygen formatting problem. Such problems exist in some other places as well. s/Occured/Occurred/ s/managment/management/ "Is there any reason to prefer the Boost.Locale implementation to the original GNU Gettext runtime library? In either case I would probably need some of the GNU tools." Maybe add a c. point about license charset_handling.html: s/does not provides/does not provide/ localized_text_formatting.html: s/when we create a messages/when we create messages/ localized_text_formatting.html: wcout << wformat(L"Today {1,date} I would meet {2} at home") % time(0) % name <<endl missing semicolon int a_and_b = a+b; please change to int a_plus_b = a+b; dates_times_timezones.html: s/they fully aware/they are fully aware/ Or rather, what does 'they' refer to here? s/than setting "current day of week"/then setting "current day of week"/ add << between some_point and " is the " s/We recommended you look/We recommend you to look/ And what date-time range does the ICU backend support? I'd like to echo other reviewers that printing date/time should not print a number by default. s/alway/always/ s/and over that build dates/and over that builds dates/ working_with_multiple_locales.html: s/get.locale_cache_enabled(true);/gen.locale_cache_enabled(true);/ s/std::locale ar=get("ar_EG.UTF-8");/std::locale ar=gen("ar_EG.UTF-8");/ s/as parameters in various functions/as parameters to various functions/ using_localization_backends.html: s/Case handing/Case handling/ recommendations_and_myths.html: s/it probably the most/it is probably the most/ s/and may other applications/and many other applications/ building_boost_locale.html: s/If your icu build in placed/If your icu build is placed/ s/if you relay on auto-linking/if you rely on auto-linking/ appendix.html: "is depend" substitute with "is dependent" or "depends" s/not bring read performance advantage/not bring real performance advantage/ s/would work only of standard facets/would work only if standard facets/ s/Boost locale fully supports/Boost Locale fully supports/ s/may broke/may break/ s/tje/the/ "including all Western, Cyrillic, Hebrew, Thai, Arabic and CJK encodings." Shouldn't be "characters"? s/std::sting/std::string/ s/not indented to be used in production/not intended to be used in production/ General remarks: Usage of comments in code snippets is inconsistent in the sense that sometimes the comment comes after the code is addresses, the other times it comes before. One of those styles should be used all over the place - preferably comments coming first as that is the common practice in other Boost libraries. Would like to see examples of using Boost.Locale facets directly. Would like to see examples of user-defined I/O inserters/extractors that respect the current state of Boost.Locale formatting as altered by manipulators. E.g. a "class money" example which uses the Boost.Locale currency formatting info for its I/O, or a "class bigint" example which uses the Boost.Locale number formatting info for its I/O. Bonus points if they fallback to std facets if corresponding Boost.Locale info is missing from the stream! Thanks, Gevorg

2 4

[locale] Formal review of Boost.Locale library starts tomorrow
by Chad Nelson 19 Apr '11

19 Apr '11

The formal review of the Boost.Locale library by Artyom Beilis starts tomorrow, April 7th, and is scheduled to last through April 16th. *************** * Its Purpose * *************** Boost.Locale provides thorough localization features to C++ programs by way of std::locale facets. Excerpted from the introduction: C++ offers a very good base for localization via the existing C++ locale facets [...] But these are very limited and sometimes buggy by design. Support for localization varies [...], and there are frequently incompatibilities between them. On the other hand, there is a great, well debugged, high quality, widely used ICU library that gives all of the goodies. But it has a very dated API that mimics Java behavior, completely ignores the STL, and provides a useful API only for UTF-16 encoded text, ignoring other popular Unicode encodings like UTF-8 and UTF-32 and limited but still popular national character sets like Latin1. Boost.Locale provides the natural glue between the C++ locales framework, iostreams, and the powerful ICU library. Although it can use the ICU library, it supports several other processing options as well. ******************* * Where to get it * ******************* You can find the "boost_locale_for_review" version here: https://sourceforge.net/projects/cppcms/files/boost_locale/ The HTML documentation can also be seen at: http://cppcms.sourceforge.net/boost_locale/html/index.html ******************** * Writing a review * ******************** The reviews and all comments should be submitted to the developers list, and the email should have "[locale]" at the beginning of the subject line to make sure it's not missed. Please explicitly state in your review whether the library should be accepted. The general review checklist: - What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? - Did you try to use the library? With what compiler? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? -- Chad Nelson Oak Circle Software, Inc. * * *

17 82

[Filesystem] error including filesystem v3 with icpc -std=c++0x
by Galen Andrew 19 Apr '11

19 Apr '11

Hello, Including boost/filesystem.hpp is giving me errors using Intel compiler in Linux with -std=c++0x that look like boost/filesystem/v3/operations.hpp(303): error: name followed by "::" must be a class or namespace name {detail::copy_file(from, to, copy_option::fail_if_exists);} ^ It compiles fine under g++ -std=c++0x. There are also a plethora of warnings like boost/filesystem/v3/operations.hpp(626): warning #279: controlling expression is constant BOOST_ASSERT(m_imp.get() && "attempt to dereference end iterator"); ^ I'd appreciate any help/ideas. Thanks, Galen

2 1