- Boost - lists.boost.org

[Unicode] Some code for y'all
by Erik Wien 31 Jul '05

31 Jul '05

Hi. I finally got a few days off this weekend, so I used them to prepare a version of the Unicode library I have written so you guys can take a look at it. This is unfortunately a snapshot of the state of things a good while back, but it's the only somewhat stable version I could find in the hurry. The basic design hasn't changed much though, so it should work just fine for getting an idea of how things work. A download of the complete library, along with a *short* (and very high-level) description to help you along can be found at the site http://unicode.irritert.net. Sorry I couldn't provide you with any in-depth documentation or stuff like that right now, but you should be able to figure it out if you really want to. ;) Anyway, Feel free to comment on anything that might spring to mind. I'll try to check back now and again to answer any questions. - Erik

1 0

Re: [boost] [Serialization] Boost regression notification (2005-07-31)
by JOAQUIN LOPEZ MU?Z 31 Jul '05

31 Jul '05

----- Mensaje original ----- De: Douglas Gregor <doug.gregor(a)gmail.com> Fecha: Domingo, Julio 31, 2005 10:06 pm Asunto: Re: [boost] [Serialization] Boost regression notification (2005-07-31) > > On Jul 31, 2005, at 12:00 PM, Douglas Gregor wrote: > > 31 failures in 1 library: > > serialization (31) [...] > > I've gone through and marked up all of these, except this one > strange > regression: > > http://tinyurl.com/ajprh > > Looking at the output of the test, I don't even seeing an error. > To > make things more interesting, the other vc-8_0 tester isn't having > any > problem. Ideas, anyone? > This looks pretty much like some spurious errors I've been observing in Boost.MultiIndex tests for this same platform: no error indication or anything. The fun part is that these errors only show every so often and then disappear, so it'll probably vanish away on the next regression cycle. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo PS (OT): Excuse my insistence, but a post of mine about an (IMHO) outdated piece of info in the getting started page have received no response: http://lists.boost.org/boost/2005/07/31129.php I can't change the thing myself, I don't have CVS access during these days. Thank you, J

1 0

CVS Locked?
by David A. Greene 31 Jul '05

31 Jul '05

The threads library seems to be iniaccessible via CVS: cvs update: [20:11:39] waiting for agurtovoy's lock in /cvsroot/boost/boost/libs/thread This has been going on for about an hour now... -Dave

1 0

Building Docs
by David A. Greene 31 Jul '05

31 Jul '05

I'm trying to write documetation for NTPs using Boostbook but I'm having trouble. The Boostbook docs say that changing to boost/doc and running bjam --v2 should build the documentation, but; >>-waller-> pwd /home/greened/src/boost/doc >>-waller-> bjam --v2 /home/greened/src/boost/tools/build/v2/tools/package.jam:72: in package.install rule stage.install unknown in module package. ../Jamfile.v2:140: in load-aux /home/greened/src/boost/tools/build/v2/build/project.jam:310: in load-jamfile /home/greened/src/boost/tools/build/v2/build/project.jam:68: in load /home/greened/src/boost/tools/build/v2/build/project.jam:111: in load-parent /home/greened/src/boost/tools/build/v2/build/project.jam:389: in initialize /home/greened/src/boost/tools/build/v2/build/project.jam:292: in load-jamfile /home/greened/src/boost/tools/build/v2/build/project.jam:68: in project.load /home/greened/src/boost/tools/build/v2/build-system.jam:80: in load /home/greened/src/boost/tools/build/v2/kernel/modules.jam:259: in import /home/greened/src/boost/tools/build/v2/kernel/bootstrap.jam:120: in boost-build /home/greened/src/boost/boost-build.jam:12: in module scope I know almost nothing about Jam so these errors don't mean much to me. For NTPs, the source uses Doxygen comments so I plan to follow the guidelines for Doxygen use with Boostbook. I've found a number of Jamfile.v2s for different libraries that seem to use Doxygen, but I'm not entirely clear how the other bits of documentation (introduction, rationale, etc.) get integrated. The Boostbook documentation is rather sparse about Doxygen use. Can anyone guide me with this? -Dave

1 0

Re: [boost] Call for interest for native unicode character and string support in boost - updated definition
by Graham 31 Jul '05

31 Jul '05

>From: Rogier van Dalen <rogiervd(a)gmail.com> >Subject: Re: [boost] Call for interest for native unicode character and string support in boost - updated definition >Hi Graham, >I fear we might be heading off in a new direction here. I think enabling third parties to add codepoints, >if desirable at all (I still have my sincere doubts) should not be a primary concern. >I propose we focus on the interface first, and leave out the implementation bits. I have been working with Unicode for many years and cannot envisage any serious Unicode implementation where private characters are not required. I have given examples: Private characters like buttons, like wrapping markers for Thai, streaming format changes, etc. etc. Each developer will need to be able to customise it - after all how can you support Unicode but decide to ignore private characters that are part of the Unicode spec ! >Discussing the implementation, I'm afraid, will just make the discussion unclear, as I'll show by pointing out some things once. >- as for the "functions" struct, do you realise that C++ has a built-in feature called "virtual functions" to do things like this? Interesting you raise this point. I started off as an interface but hit a major problem - performance. I tried wrapping 100,000 words and the interface took a hell of a hit due to the way in which the compiled code takes the interface and then offsets to the member to call a function on the interface. So I looked at being able to access the function on the interface directly - this proved to be impossible via a static function and you cannot have statics being implemented on an interface and no better if you wrap an interface call using mem_fun. Then I considered how I would modify one function, like sort, without ANY performance hit - and ended up with what I had. How the hell can you do it more neatly without a performance hit? I am open to suggestions. Do not forget we must be able to optimise this for times when this interface will be hit millions of times per second - and yes this is something I have done a number of times in real life - e.g. formatting of live data feeds - and this must be a primary concern in the design. - Have you realised what happens when deque<>::iterators, or any other iterators to containers with non-contiguous elements, are fed into the get_uppercase method with the current implementation? What about iterators that process UTF-8 and pretend it to be a UTF-32 sequence? The implementation will work on any non-contiguous iterator as written, after all the single iterator version increments/ decrements the iterator then de-refs it. Of course from bitter experience I can tell you that working with a single iterator and not caching the values is a massive performance hit. - Do you realise that inline functions are not macros and thus need no backslashes at the end of lines? Sorry - I went through a number of iterations to try and get the code you saw - in one of the iterations I tried to inline the functions. >The implementation won't help discussing the interface, so I think we'd better leave them out for now I am actually finding that the snapshots I have included are important as they can help to reveal why the implementation might not work and what we can do about it. > >BTW, now looking at the code: do you fully realise how iterators work? >It seems to me that >StartOfGrapheme(functions* pFns, inputIterator chPrev, inputIterator ch, inputIterator chNext) is not really needed, >because it is quite easy to find an iterator to the next and previous position given the current one. That is correct(ish) - but processor intensive - I have had real problems when you process a document from a data feed and wrap it for storage in a database. As an example on a ".4 Gig Pentium 4 I have seen this take up to six seconds [depending on the text - and we were handling some large text documents] when fully optimised - and that required a lot of profiling to get the best performance. Wrapping a full document for the first time requires a lot of processing and you cannot just increment and decrement iterators - as I said in my previous e-mail [the one today, after the e-mail to which you have replied] the only way of doing this with reasonable performance is to have three iterators that get shuffled along. Remember these are really low level functions and get hit very heavily and often. >Please find attached a modified version of the header. Changes are: >- It now looks like C++: superfluous semicolons are deleted and identifiers are lowercase with underscores; This will mean altering the automatic conversion of the Unicode names to this form C++ names but this should not be a problem stripping out the upper case a replacing spaces and - with _. >- the last "uni" prefixes have been removed; >- it does not show the implementation any more; Unfortunately you seem to have fallen into the great trap of iterators. The code can no longer be used to/ from third party DLLs, please see my post of earlier today. Iterators do not work unless you have all the source code - which means that third party DLLs containing custom controls will not work with your changes. This would be a major step backwards as grid controls are a major business and many are shipped without source, just as DLLs with a defined interface. You cannot use an iterator across such a boundary - please see my previous code for an example of how I have ensured that it WILL work with third party DLLs such as grid controls without every DLL from a different company having to have it's own Unicode data. The implementation shown demonstrated how to do this and was included for that reason. >- I corrected some spelling errors, possibly introduced new ones (seperable is spelled sepArable, isn't it? I'm not 100% sure); >- a get_category() function is defined to get the general General Category (i.e. letter, mark, number, etc.) I have just realised that all the category enums need to be placed in a single enum as they are in fact in a single data space so your get_category becomes the single category call. >- I deleted the page0() function because I didn't see why it should be there (feel free to move it back in if I missed something). OK >- I provided only iterator-based grapheme, word and sentence skipping functions; This will not work across third party code when you don't have the source. Means you cant have any more grid control DLLs etc in Unicode unless they contain all the Unicode data separately or you insist that all suppliers have to sell their DLLs as source code. >- I provided a locale with a few lines sketching an idea of what a collation object should look like. Did you realise that string comparison should likely be passed to a function or >container as a comparison object? For example, my specification would allow: > >// Some container c of strings >// Some string s >std::lower_bound (c.begin(), c.end(), s, unicode::default_locale().collate_accents()); > >Any comments are, of course, most welcome! We need to make a collation object that is container independent as there are going to be several Unicode containers - not just one. We must therefore separate collation from container. It must work with third party DLLs where you don't have the source code so iterators are out. The collate_accents call is strange - I don't understand why you don't just pass a locale in. For example some languages sort <ae> differently and this is not an accent. I would therefore just pass in a uint32_t for the locale. Any thoughts? Yours, Graham

2 1

[1.33.0] Where is Logo???
by Pavel Chikulaev 31 Jul '05

31 Jul '05

Hello boosters, We only have some hours before release but the logo we all chose is not on the home page yet... What's happening? Or that logo was chosen for 1.34? -- Pavel Chikulaev

2 1

[regex] begin-/end-line need to be customizable via traits
by Eric Niebler 31 Jul '05

31 Jul '05

I think there is a problem in the regex standardization proposal regarding the begin- and end-of-line assertions (^ and $). It seems that there is no way to customize their behavior via the traits. This is inconsistent with the word boundary assertion, which is implementable in terms of the word character class (\w). The author of a traits class should be able to specify which characters are line separators. A simple fix would be to add a character class for line separator characters. Then, ^ and $ could be implemented in terms of lookup_classname and isctype, just as the word boundary assertion is. This leaves out an important corner case, though: \r is a line separator only if it is not immediately followed by a \n. I haven't yet come up with a traits interface that clean enough and general enough to satisfy. I'm open to suggestions. -- Eric Niebler Boost Consulting www.boost-consulting.com

2 1

[Boost-bugs] [ boost-Support Requests-1248497 ] borland builder 6 compiler error
by SourceForge.net 31 Jul '05

31 Jul '05

Support Requests item #1248497, was opened at 2005-07-31 14:17 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=207586&aid=1248497&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: hamid r. najafi (hrnajafi) Assigned to: Nobody/Anonymous (nobody) Summary: borland builder 6 compiler error Initial Comment: I want used BOOST in my projects. But I can't compile BOOST with Borland Builder 6. For example I'm compiled (ROOT)\libs\graph\example\print-adjacent- vertices.cpp and compiler reports more than 26 error. please see attached file. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=207586&aid=1248497&group_… ------------------------------------------------------- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click _______________________________________________ Boost-bugs mailing list Boost-bugs(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/boost-bugs

1 0

Re: [boost] Call for interest for native unicode character and string support in boost - updated definition 3
by Graham 31 Jul '05

31 Jul '05

Dear Rogier, >> What do you think? I am happy to go to uint16_t and limit to 4 levels as >> this is a reasonable performance/ implementation trade-off that I have >> previously used with success. >I hadn't thought of that, actually. But as for the sort keys: I was >actually thinking of packing the key *with* the string, so that the >string itself can be used as a tie-breaker (cos that's what you need >the 32 bits for, isn't it?). >class string_with_sort_key >{ > const string s; > std::vector <uint16_t> sort_key; >public: > string_with_sort_key (const string &); >}; >This would make a good key for, say, std::map: it woul implicitly >converting from string when required, e.g. when calling "find" with a >string. I think the string itself would be needed in case another >locale is required, and as a tie-breaker, but I haven't fully thought >this through yet. What do you think about this? Yes - almost exactly what I had in mind. We can then use the sort data to level 4 with uint16_t and the string is present. The final flourish might be to have a version that has the canonical decom of the string present as well for times when you hit a sort tie. I have never needed to handle a sort tie in this way, but we have the flexibility under this scheme to even have two classes string_with_sort_key and string_with_sort_key_and_decomp as this is just a wrapper round the actual processing call and developers can then use whichever they prefer for the memory/ complexity tradeoffs on their system. >> >done without producing sort keys. >> >> String equality [=] would be handled directly using on the fly >> comparison of canonically decomposed forms as I previously mentioned, >> and as you correctly mention. >> >> String equivalence [<] would be handled using sort level 4+. >I'd rather use binary comparison for the normal operator< and >operator==: that's what the Unicode standard recommends for speed >(5.18). For UTF-8 and UTF-32 it's as fast as normal string comparison, >and for UTF-16 it's only slightly slower. C++ has this habit of >passing comparison objects (like std::less) to algorithms and >containers that would nicely fit in with the need for locales. >Basically, a "collate" object would contain the information needed to >compare strings for a given locale on a certain level. Fine - I don't have a problem with it - the only reason that I have prevented it in the past was to ensure good programming practice [i.e. force use of the collation version] at the cost of removing the simple version. >I'm not sure I see what you mean, sorry. I'd probably have to see the >code. If I am correct, however, for advancing to the next grapheme >boundary you don't need a "previous" iterator, because it only has a >look-back (if that's the antonym of lookahead) of one character max. I used to think that I only needed two characters [current and next] for grapheme testing but then I hit some oddities. To do a reliable test you actually do need the previous character for: <cr><lf> as opposed to <lf> i.e. 123<lf>, the <lf> is a grapheme start but 123<cr><lf> the <lf> is not a grapheme start Hangul syllable sequences Base char + virama + letter And a few of the others... >Whoops... I'm only now starting to see that we probably have slightly >different headers in mind: I'm thinking of what the user needs, you're >thinking of what the "database" needs... sorry I didn't realise that. >Obviously, the DLL shouldn't use iterators, and should probably have, >well ermmm, just about the interface you propose. Hopefully this >realisation will get us somewhere because I'm sure any iterator type >can be used for processing, even if the database returns plain >pointers. It'll probably mean the database interface should be as >low-level as possible and any smarts should be outside it. These >should definitely be different header files, too... >For example, I think the database (DLL) should provide the >Grapheme_Cluster_Break property for every character; the actual >algorithm could be templated, allowing any iterator to be used. Great - this is an extremely important breakthrough in mindset - you have a fixed interface for sharing round third party DLLs etc, but can write templated iterators around the interface calls. >> ... >> > (I don't think "next" is >> needed for the grapheme_cluster case anyway.) > >Correcting myself: I meant to say "previous" rather than "next", but see up. Moving a cursor left requires previous grapheme. >> Does this mean that sorting will also require a locale? > >In my latest header proposal, I've grouped both functions under one >"locale" object, and I think this would be fast, allow for (later) >adding tailoring for language, and yield a reasonable syntax. What do >you think? What the implementation does internally and whether it decides to do use a locale class should not be specified at this level. I think it would be simpler to just pass a locale ID in, and then let the implementation move the few exceptions around dependent on the locale using whatever internal technique it wanted. Let me know what you think of the latest iteration that shows the interface into the character implementation and around which we will probably added templated inline calls for iterators etc. Yours, Graham

1 0

Re: [boost] Call for interest for native unicode character and string support in boost
by Graham 30 Jul '05

30 Jul '05

>From: Rogier van Dalen <rogiervd(a)gmail.com> >Subject: Re: [boost] Call for interest for native unicode character and string support in boost Dear Rogier, >> DWORD is actually uint32_t. >I know, but - correct me if I'm wrong - I think the sort keys consist of 16-bit integers. This depends on how we implement it - and if we implement or allow for more than 4 level comparison - if so the 5+ comparison if coded may require uint32_t. What do you think? I am happy to go to uint16_t and limit to 4 levels as this is a reasonable performance/ implementation trade-off that I have previously used with success. >> I believe collation must be here as there will be probably be several >> containers with Unicode characteristics and this is a good level for >> them to work on. >I'm not sure - seeing as I'd expect us to come up with at least the option to keep strings in some Normalisation Form, the >first step of the collation algorithm should be skipped in some cases. Furthermore, individual comparisons should probably be >done without producing sort keys. String equality [=] would be handled directly using on the fly comparison of canonically decomposed forms as I previously mentioned, and as you correctly mention. String equivalence [<] would be handled using sort level 4+. >Not being a native speaker of English I don't think I understand what a "coding hit" is. Reduction in performance - your English is better than a lot of the British ! >I'm assuming that the caller usually knows the start and end. (I'm obviously assuming a C++-style string and not a C-style >zero-terminated one.) In this case, the caller must check whether the end has been reached and whether it's at begin, >pass in 0 in those cases, and then start_of_grapheme has to check whether 0 was passed in. To me it >seem this would be slower, both to code and to execute. Performance can be argued back and forth, but when I looked at optimising the performance I needed three iterators as iterating graphemes is not a 'cheap' operation - and I moved the iterators along in series [step forward is curr->prev, next->curr, calculate next]. In 64 bit code iterating a uint_32t container, iterators can introduce high costs. The real problem though is that iterators do not work across DLL boundaries unless you have the code for the DLL and the caller. Have a look at my recently posted implementation. This implementation proposal would mean you can produce a DLL for sale that has a custom Unicode control in it - without having to have the source code for the iterators that it needs and providing an interface for each iterator type, or placing the whole of the Unicode data in that DLL and preventing the use of custom characters. >Furthermore, one'd better abstract away from the implementation as much as possible, and I don't think it is relevant to the caller >how far the functions want to look ahead. (I don't think "next" is needed for the grapheme_cluster case anyway.) A good example of next grapheme and start of grapheme is for default moving a cursor left/ right - the cursor may be on a grapheme [or optionally inside a grapheme if the editor supports this, as is optionally recommended in the standard] and you need to find the previous/next grapheme. >Ermmm... what's this doing then on p. 137 of Unicode standard 4.0.0? >"Characters may have case mappings that depend on the locale. The principal example is Turkish, ..." Ouch - I had not seen that before. I believe that this would mean that the upper/ lower case conversions will need to take a locale to allow for these special extensions at a later date. I have not seen any written rules for this - I guess because I had never worked with Turkish - but it should be allowed for. Does this mean that sorting will also require a locale? Please let me know what you think and if you can spot any other assumptions I have made in my attachment posted on the 28th. Yours, Graham

2 1