- Boost - lists.boost.org

Re: [boost] [rfc] Unicode GSoC project
by Graham 15 May '09

15 May '09

>I'll be reviewing it in a while. >I'm not too sure about the memory layout it uses (__uni_char_data could >really be compressed to use less memory for example), nor about the >interface it exposes, but it does seem to work well. >About is_grapheme_break though, isn't the implementation for legacy >grapheme cluster rather than extended ones though? Dear Mathias, The uni car data was laid out in that way for speed and simplicity. Please feel free to do it better. The grapheme break worked off the character attributes and standard rules. If it needs changing or did not cover a later standard, then it should be updated. The principle would not change as it makes the string iterators quite straightforwards.

1 0

[rfc] Unicode GSoC project
by Mathias Gaunard 15 May '09

15 May '09

Hi everyone. I'm in charge of the Unicode Google Summer of Code project. I have been working on range adaptors to iterate over code points in an UTF-x string as well as converting back those code points to UTF-y for the past week and I stopped working on these for a bit to put together some short documentation (which is my first quickbook document, so it may not be very pretty). This is not a documentation of the final work, but rather that of what I'm working on at the moment. I would like to know everyone's opinion of the concepts I am defining, which assume the range that is being worked on is indeed a valid unicode range in a particular encoding, as well as the system used to enforce those concepts. Also, I put the normalization form C as part of the invariant, but maybe that should be something orthogonal. I personally don't think it's really useful for general-purpose text though. While the system doesn't provide conversion from other character sets, this can easily be added by using assume_utf32. For example, using an ISO-8859-1 string as input to assume_utf32 just works, since ISO-8859-1 is included verbatim into Unicode. The documentation contains as well some introductory Unicode material. You can find the documentation online here: http://mathias.gaunard.emi.u-bordeaux1.fr/unicode/doc/html/

7 16

[C++0x] More config macros needed
by Beman Dawes 15 May '09

15 May '09

GCC 4.3 and even more so 4.4 have implemented significant portions of the C++0x core language, and provide lots of C++0x standard library components too. Microsoft 2010 beta 1 will ship "real soon", and it too will have some major C++0x core language features, and also some C++0x standard library components. Thus we need more config macros, particularly those aimed at the standard library. A possible approach: BOOST_NO_CONCEPTS // needed to tell if headers have been conceptized BOOST_NO_*_HDR // for each new C++0x header; * is name of header For example, BOOST_NO_CHRONO_HDR is defined unless header <chrono> is present. I'm proposing we not try to identify whether or no the header is complete. If BOOST_NO_CONCEPTS is defined, the header obviously hasn't been conceptized. Ditto anything else that depends on any other new core language feature that hasn't been implemented. Other incomplete aspects will have to be dealt with by testing the library version. We don't want feature macros to proliferate unduly; they are more work to create and maintain than you might think, and if there are a lot of them it gets very confusing. Comments? --Beman

3 6

[Preprocessor] Can we generate token for #include programmatically ?
by Joel Falcou 15 May '09

15 May '09

Here is what I had in mind #define BASE foo.hpp #include BOOST_PP_STRINGIZE(BOOST_PP_CAT(nt2/support/numeric/functions/scalar/,BASE)) It fails because it seems we can't paste stuff ending with / ... Using fiel iteration, how can I circumvent hsi problem and generate a list of #include whose filename are computed from a sequence of name in a BOOST_PP_SEQ and a given, changing, prefix path ?

2 6

Re: [boost] [rfc] Unicode GSoC project
by Graham 15 May '09

15 May '09

Dear Eric/ Mathias, >That's good, these are needed. Also needed are tables that store the various >character properties, and (hopefully) some parsers that build the tables >directly from the Unicode character database so we can easily rev it >whenever the database changes. A good reloadable character library is in the vault. >And UnicodeGrapheme concept doesn't make sense to me. You say, "A model of >UnicodeGrapheme is a range of Unicode code points that is a single grapheme >cluster in Normalized Form C." A grapheme cluster != Unicode code point. It >may be many code points representing a base character an many zero-width >combining characters. So what exactly is being traversed by a >UnicodeGrapheme range? >It is thus important to be able to apply algorithms with graphemes as the >unit rather than code points to deal with graphemes not representable by a >single code point. I think that a grapheme is more of an iterator concept than a data type concept. By specialising it you will unnecessarily complicate any library. Don't forget that, for example, the current grapheme may start as one character, then suddenly 'grab' the surrounding characters as it makes a combined glyph. I have never found a use case in practise where specialising the grapheme as other than a validated series of code points was helpful. The two cases where graphemes are important is in display [which requires intermediate glyph conversion anyway, and works just as well on runs of code points, so code points are fine] and in editing - and the grapheme-ness here alters during typing. >The Unicode standard also specifies various features such as a collation >algorithm in Technical Standard #10 - Unicode Collation Algorithm for >comparison and ordering of strings with a locale-specific criterion, as >well as mechanisms to iterate over words, sentences and lines Have a look at the character library that I posted in the vault - if you can do graphemes then you can do words, paragraphs etc as they are all just attributes of the characters with simple rules. Graphemes come in to their own for text display and editing and you would need these as well to be able to support that. Don't forget that windows GDI only supports point arithmetic and this means that you need to be able to locate word boundaries to display text well at different scales to work around the GDI scaling rounding [and GDI+ is not much better].

2 1

[1.39.0] Beta candidate
by Beman Dawes 15 May '09

15 May '09

The beta candidate files are up at http://boost.cowic.de/rc/ Before pushing these out to SourceForge, it would be great if a few Boosters could download and try them out as a final QA check. Post your experiences here. Thanks, --Beman

9 24

Review Request: Algorithm.Sorting
by Steven Ross 15 May '09

15 May '09

I have uploaded algorithm_sorting.zip for a formal review to become Boost.Algorithm.Sorting, if people feel that is appropriate. It is 3 templated hybrid radix-based/comparison-based algorithms, specialized to sort certain types of data more efficiently. All 3 have superior worst-case and average-case performance to std::sort, based upon my testing. None of these algorithms use much additional memory. All of them can sort key + data, and as they do fewer swaps, tend to do so much faster than std::sort. As a general rule, the speedup of these algorithms relative to std::sort increases as data set size increases. These algorithms tend to be faster relative to std::sort when comparisons are slow, as they do less (or no) comparisons. string_sort, which sorts strings in O(n*length) time, as compared to O(n*length*log(n)) for pure comparison-based algorithms. It is consistently faster than std::sort in string sorting, by a little more than a factor of 2 based upon my testing. float_sort, which sorts IEEE floating-point numbers as integers, but attaining the same ordering as std::sort, with the exception of -0.0 and NaNs, both of which cannot be ordered based upon their comparison function. float_sort is roughly twice as fast as std::sort on UNIX-type OSes I've tested on, and roughly 7X as fast on Windows. I believe this speedup to be because floating-point comparisons are slow on Windows. integer_sort, which sorts integers around 60% faster than std::sort on UNIX-type OSes, and 25% faster on Windows. On Windows with MSVC, std::sort sorts already-sorted and small range data extremely quickly, and integer_sort processes already-sorted data slower than it by as much as a factor of 3X. I believe this to be due to some optimization for already-sorted data, as integer_sort takes substantially less absolute time on already-sorted data than unsorted data. So integer_sort on Windows has a tradeoff: better worst-case and average-case performance, but worse best-case performance. integer_sort and float_sort share some tuning parameters. I have set values that seem to work well on both UNIX-type and Windows OSes, but people are welcome to try to see what values are best for their system and problem type. The ideal values for integers and floats may be different on some systems (such as Windows), but for simplicity they are kept the same. A tuning/testing script named tune.pl is included, for those who wish to see how fast these algorithms are on their system (and verify correctness). I don't recommend using it for tuning without the -large option, and that takes days. If you do use it for tuning, I also recommend the -verbose option, to see what it's doing.

6 19

[exception] current_exception mishandling standard exceptions
by Adam Badura 14 May '09

14 May '09

It seems that current_exception mishandles standard exceptions. I noted two problems: 1) Some standard exceptions are not handled at all. This includes std::runtime_error and its derived (std::overflow_error, std::range_error, std::underflow_error), std::domain_error (handled by general std::logic_error), std::length_error (handled by general std::logic_error) and std::ios_base::failure (handled by general std::exception). Why is it so? 2) Non-standard exceptions derived from standard exceptions are sliced by current_exception. It detects those types by throwing and caching and thus will catch also derived types (like null_pointer : public std::invalid_argument). But during construction of internal structures the standard exception type is used and thus copy construction slices the original exception object. If typeid was used instead of throwing/catching (or dynamic_cast) to detect the type current_exception could at least detect that is will slice the original object and use a different type so it would not pretend everything is OK. Adam Badura

3 6

[property_tree] XML parser broken in 1.39.0
by Boris Schaeling 14 May '09

14 May '09

Calling boost::property_tree::xml_parser::read_xml() to parse a XML file and populate a property_tree doesn't work anymore in Boost 1.39.0 (at least not in Visual C++ 2008). The compiler error I see is: boost\iterator\iterator_adaptor.hpp(230) : error C2027: use of undefined type 'boost::STATIC_ASSERTION_FAILURE<x>' with [ x=false ] The property_tree headers include boost\spirit\home\classic\iterator\position_iterator.hpp which includes boost\iterator\iterator_adaptor.hpp. I also see this warning: Warning: This header is deprecated. Please use: boost/spirit/include/classic.hpp Warning: This header is deprecated. Please use: boost/spirit/include/classic_position_iterator.hpp It looks like as if a change in Boost.Spirit or Boost.Iterator broke the XML parser. Does Boost.PropertyTree have a maintainer who can look into this? Boris PS: As it turns out it's broken in Boost 1.38.0, too. With 1.36.0 I don't get a compile error (1.37.0 I don't have installed and can't test).

3 5

Re: [boost] [rfc] Unicode GSoC project
by Graham 14 May '09

14 May '09

> 1. The vast majority of characters (by frequency of use) are > on the BMP. > 2. For seamless integration with the majority of existing > software with good Unicode support. > >I don't find either of those claims very convincing. I hope that your >library will not try to make UTF-16 some sort of default encoding, or >otherwise give it special treatment. Phil, I think each application needs to be assessed on its own merits. For example: For data storage on mainly ASCII text I use UTF-8. For text manipulation I often use UTF-16. For a Unicode text editor I use UTF-32. There is not a one size fits all. Yours, Graham

1 0