- Boost - lists.boost.org

Tokenizer doc problems/confusion (?)
by David Abrahams 07 Apr '04

07 Apr '04

1. http://www.boost.org/libs/tokenizer/tokenizer.htm says: template < class TokenizerFunc = char_delimiters_separator<char>, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ class Iterator = std::string::const_iterator, class Type = std::string > class tokenizer Yet char_delimiters_separator is officially deprecated. Is that really intentional? Wow, it appears to be using the deprecated class template for the default! Now, I wanted to tokenize an input stream, without putting it in a string first. It seems to be much harder than neccessary: #include <map> #include <string> #include <iostream> #include <boost/tokenizer.hpp> #include <boost/lambda/lambda.hpp> #include <iterator> int main() { typedef std::map<std::string, unsigned> fmap; // Seems awfully complicated boost::tokenizer< boost::char_delimiters_separator<char>, std::istreambuf_iterator<char> > t( (std::istreambuf_iterator<char>(std::cin)) , std::istreambuf_iterator<char>() ); fmap f; std::string s; using namespace boost::lambda; std::for_each(t.begin(), t.end(), ++var(f)[_1]); for (fmap::iterator p = f.begin(), e = f.end(); p != e; ++p) std::cout << p->second << ": " << p->first << "\n"; } I can think of lots of ways to simplify the interface, most of which center on eliminating redundant mentions of istreambuf_iterator<char>. When I throw the following text at it: ------ how much wood could a woodchuck chuck, if a woodchuck could chuck wood? ------ I get: 2: a 2: chuck 2: could 1: how 1: if 1: much 2: wood 2: woodchuck as desired. But if I replace char_delimiters_separator with char_separator, I get: 15: What's up with that?? Even if char_separator did what it was advertised to (and it's not clear that it does), it wouldn't give me the simple "find the words functionality" of char_delimiters_separator... so I'm baffled by the deprecation. 2. http://www.boost.org/libs/tokenizer/char_separator.htm says: explicit char_separator() The function std::isspace() is used to identify dropped delimiters and std::ispunct() is used to identify kept delimiters. In addition, empty tokens are dropped. which seems strange in light of the fact that there's no ctor taking _functions_ to be used to determine kept/dropped delimiters, and nowhere in the text do you indicate that functions are called internally. Help? Thanks, -- Dave Abrahams Boost Consulting www.boost-consulting.com

1 0

RE: [boost] Generic non-derivable base class
by Reece Dunn 06 Apr '04

06 Apr '04

Matthew Vogt wrote: >The following simple base class implements the mechanism as dicussed on >Stroustrup's FAQ page, and encapsulates the virtual inheritance so that the >class can be non-virtually inherited, removing the most obvious chance of >implementation error. >Will this code work on multiple platforms? I have only tested on gcc >3.2... >[snip] I have tested on MS VC 6.0 sp5, 7.0 and 7.1 without any problems. On Borland 5.5.1 however, I am getting the following errors (tested in my errorcheck component): [log] Error E2404 d:\devel\adl++\include\boost/errorcheck.hpp 23: Dependent type qualifier 'boost::detail::nonderivable_base<boost::errorcheck<boost::policy::error_storage<int>,boost::policy::no_errormsg_storage> >' has no member type named 'identity' in function main() Error E2061 d:\devel\adl++\include\boost/errorcheck.hpp 23: Friends must be functions or classes in function main() Error E2029 d:\devel\adl++\include\boost/errorcheck.hpp 33: 'boost::detail::nonderivable_base<boost::errorcheck<boost::policy::error_storage<int>,boost::policy::no_errormsg_storage> >' must be a previously defined class or struct in function main() Error E2029 d:\devel\adl++\include\boost/errorcheck.hpp 136: 'boost::nonderivable<boost::errorcheck<boost::policy::error_storage<int>,boost::policy::no_errormsg_storage> >' must be a previously defined class or struct in function main() [/log] I don't have any workarounds at present. Regards, Reece _________________________________________________________________ Sign-up for a FREE BT Broadband connection today! http://www.msn.co.uk/specials/btbroadband

1 0

Re:[boost] Re: FC++ -- Formal Review Result
by Robert Ramey 06 Apr '04

06 Apr '04

David Abrahams wrote >> Andy Little wrote >> > Whether anyone has the guts to employ you I have no idea, but I >> > wish you all the best for the future. > Condescension and insult. FWIW - I read that as a complement. IMO many employers ARE reluctant to hire out of the box thinkers. Of course my view may be skewed as I've been a freelancer for 35 years. Robert Ramey

1 0

RE: [boost] Formal Review: Assignment Library
by Powell, Gary 06 Apr '04

06 Apr '04

Either I missed it or, I'm confused, why is the make_insert functions for set & map using the "hint" of c.end() for the insert? Is there any advantage to this? -Gary-

2 1

FC++ -- Formal Review Result
by Mat Marcus 06 Apr '04

06 Apr '04

The formal review for FC++ has completed. I am sorry to say that FC++ is not accepted into boost at this time. It may be possible for the library to be accepted in the future, but not without changes substantial enough to warrant an additional review. I will describe some possible strategies for later acceptance together with the rationale for my decision. Note that while the library is not accepted, the author has successfully established some of the merits of the functional programming paradigm and we should not require him to re-establish those same points in a future review. For that matter, I'm not convinced that we should have asked him to establish these points in the first place. * Rationale This was a very difficult decision, and I spent a long time sitting on the fence. I believe that the functional programming paradigm is important for C++ programmers, and that there are many interesting features available in FC++ such as lazy lists and the standard prelude functionality. In addition, though there was not a consensus amongst reviewers to accept (see Opinions of the Reviewers below), several boost library authors offered positive reviews of FC++. I finally managed to gather my thoughts and come to a decision -- thanks to all for your patience. In formulating my decision, I realized that one of the important questions to ask of a candidate boost library is: "Can I in good faith recommend to any C++ programmer with interest in domain X that they spend the time and effort to learn library Y and consider using it in relevant projects?" For FC++ I can not yet in good faith answer yes. I do think that it is worthwhile for *some* programmers to look at and would certainly consider pointing to FC++ in its present form from the boost web site as an "interesting third party library" if we maintained such links. But as I said I am not yet ready to put the boost "brand" on FC++. Why not? My evaluation was based on two main criteria, practicality and educational value, of which detailed accounts follow. ** Evaluation of Practicality Today FC++ is a largely monolithic library with a fair degree of overlap with other boost components. This led to a good deal of confusion/discussion during the review period. Such a state of affairs (monolith with substantial overlap) could conceivably be acceptable if there were clear problems for which FC++ was a uniquely suited practical solution. Exact Reals (see footnote *** below) notwithstanding, I don't believe that the *practical* value of FC++ as a *monolithic* embedded DSL in C++ supporting mixed paradigm programming has been satisfactorily established. [footnote ***: The existence of clients such as the Exact Real lib help demonstrate some degree of practicality. During the review I briefly examined the Exact Real site. This looked like an interesting application though I could not yet say how much practicality was demonstrated since, for example, the performance characteristics of lazy lists and constructive reals were not immediately clear to me. If FC++ is to be accepted on practical grounds potential users may want to be able to understand performance characteristics.] If the authors were to break FC++ into smaller pieces and should they participate in a future lambda, phoenix, FC++ merger then the above concerns would be addressed. It surely doesn't hurt that the authors of those other libraries expressed strong interest in the above direction. And as was pointed out in the review some of the ideas from FC++ might successfully be applied to other boost libraries such as function and/or bind. If FC++ was part of a single consistent set of FP components within boost then there would be good odds of achieving consensus to accept. The changes to bring this about would be of significant enough magnitude however that I would not be comfortable with provisional acceptance today -- a future review (or reviews of pieces are submitted independently) would be required. ** Evaluation of Educational Value Since boost is not necessarily restricted to purely practical libraries I also considered accepting FC++ into boost on the grounds that it provides an environment in which C++ programmers can learn more about and experiment with FP. FC++ enjoys some degree of success in this area. However I found it disturbing that during the review a number of traditional/imperative-style C++ programmers were just not "getting it" about FP. Perhaps a case could be made for accepting FC++ in its entirety on educational grounds after some of the improvements suggested by reviewers were implemented, e.g. improvements to the documentation and naming (useful in any case), the completion of some additional features such as Monads, or deeper documentation of the useful implementation techniques. But I must admit that I find this strategy less attractive, and I would urge you to try to break things up instead. One of the difficulties in finding acceptance on purely educational grounds might be characterized like this: if I was to recommend a way for an imperative programmer to learn more about functional programming today I would probably suggest a jump into an (OCa)ml, Haskell, or even a Lisp/Scheme environment. I am not sure that this I would recommend the FC++ environment for this purpose even with the changes suggested above. That is, I am not entirely convinced of the *purely educational value* of working with a functional programming DSL within C++. I really would prefer to see the FC++ ideas used in a practical mixed paradigm setting. * Opinions of the reviewers I was a bit concerned about the dearth of reviews during the first week so I decided to extend the review. The final number of reviews was still lower than I would have liked, but in the end I received approximately 4 no votes and 7 yes votes, with varying amounts of effort spent. Of the no votes, there was some support for acceptance after a later review, pending certain improvements. At the same time a number of the yes votes urged acceptance only after similar concerns were addressed. From this one could argue that there was near consensus that the community would be interested in accepting FC++ after these issues were addressed. The issues include the state of the documentation, the monolithic nature of the lib, and the incomplete nature of some components of the lib, e.g. monads. So I view some of the yes votes and no votes as differing mainly on whether to accept now with the above provisions or whether to require a second review after the above concerns were addressed. I spent some time vacillating between those options before finally settling on my decision to require a second review before acceptance. Given the significant nature of the requested changes I was only comfortable choosing the latter option. * Conclusion Though I am sorry not to be able to accept FC++ into boost today I would like to acknowledge its authors for the influence this library has had on other boost components. I strongly encourage you to keep working to bring the innovations from FC++ into boost in the future. I would also like to acknowledge Brian for his many creative contributions to the boost community and his thoughtful and professional posts. I am sure that I am not alone in saying thanks for all your efforts and I look forward to hearing more from you in the future. - Mat

9 14

RE: [boost] Re: [program_options] Re: Unicode support
by Ferdinand Prantl 06 Apr '04

06 Apr '04

Hello, > From: Vladimir Prus [mailto:ghost@cs.msu.su] > > > glib did a very good > > implementation of UTF-8 handling and Glibmm is a well done > C++ wrapper > > but it lacks the "standardness". Something like > boost::ustring COULD > > bring a widely accepted UTF-8 aware unicode string to C++ > programmers. > > A somewhat relieving thought. > > I am not exactly sure if UTF-8 or UCS-4 is better as > universal solution, but some solution is surely needed. I am afraid there is no universal solution for all users. The easiest solution is based on the native basic_string<>, which is specialized for char (8-bit) to support ASCII/ANSI encodings and for wchar_t (16-bit) usually used for UCS-2 encoded strings. UCS-4 (32-bit) encoding would require another basic_string<> specialization. UCS-2 held all characters in Unicode 1.1, There was a need for more unique numbers and UCS-4 was introduced in Unicode 2.0. Unfortunately there is no 4-byte character specialization for basic_string<> in STL yet. Generally speaking UTF encodings (e.g. UTF-8 or UTF-16) are not suitable for the fast in-memory usage because some operations, for example at(int) and length() have not O(1) but O(n) to give a result. They (UTF, T=Transformation) are better for storing texts as they save place by using variable number of bytes for a character. However UCS (e.g. UCS-2 or UCS-4) are fast for memory operations because they use fixed character size. That is why I would not like to use basic_string<utf8char> in memory, rather basic_string<ucs4char> instead, but I would not generalize it for all possible applications. You can expect initialization from (const char * argv []) on all platforms or (const wchar_t * argv []) on Windows in UCS-2. With the basic_string<> you already have support for the parameters from the current locale (char) and for parameters in UCS-2. If we take an option to read parameters into basic_string<wchar_t> or basic_string<ucs4char_t>, where the character size or encoding is not the same as the native encoding on the command line, there is an affinity to streams. Some shells allow usage of UTF-8 encoded parameters or, generally, usage of characters out of the current locale. It means, that a program can choose the way, how to encode all characters from Unicode to char. UTF-7/8, etc. I would like to have a solution similar to streams: imbue(). Having this, you could convert internally every argv[x] using imbue(y) applied to a stringstream, where the facet y provides the caller. The target character capabilities could choose the caller by providing a basic_string<> specialization. On the other hand, such a conversion can be performed also by a user. The parameters sent to main() are char* or wchar_t* and thus program_options can give them back just as they are in basic_string<char> and basic_string<wchar_t>. The client can use his facet to imbue a basic_stringstream<> initialized with the parameter from program_options. Or a conversion library could be used to perform the conversion, something like lexical_cast<> does for type converions. It is a matter of convenience only - separate converions library (no support for encoding in program_options) or imbue(), performing the conversion inside the program_options. However, the conversion should not be implemented for program_options only, that is why I suggested an existing interface - facets. > > Or did I miss something? Is something like this part of > boost already? > > Nope :-( Even UTF-8 encoder is not in boost yet. You can find come converting facets for UTF-8 raedy for imbue() to a stream in the files section on yahoo. Unfortunately not finished or not reviewed... Ferda > > - Volodya

4 3

Planning Next Release
by Aleksey Gurtovoy 06 Apr '04

06 Apr '04

Hi all, I'd like to test the waters for the last week of June -- let's say Monday, June 28 -- being the target date for the next major release, which I am volunteering to manage. Correspondingly, June 21 or so would be a branch-for-release date, by which everything should be in CVS and in the releasable shape. What is a "releasable shape" is a separate question I'd like to postpone for now, but basically the proposed target date implies that the for all the new stuff to get into release it'd have to be in the CVS by the middle of June -- which gives us about two months to prepare/finish things up. How does this sound for everybody? Any objections/adjustments to the dates? -- Aleksey Gurtovoy MetaCommunications Engineering

6 7

How to get an unique identifier for an edge
by Lodewijk Smit 06 Apr '04

06 Apr '04

Hi, I tried to get an unique identifier for a node and an edge. The node is possible e.g.: adjacency_list<> someGraph; std::pair<adjacency_list<>::vertex_iterator,adjacency_list<>::vertex_iterator> p; ed = add_edge(0,1, someGraph); property_map< adjacency_list<> , vertex_index_t>::type in; in = get(vertex_index_t(), someGraph); for (p = vertices(processGraph); p.first != p.second ; ++p.first) { cout << *p.first << "\n"; cout << "index: " << in[*p.first] << " \n"; } However, the same idee does not work for an edge: std::pair<adjacency_list<>::edge_iterator,adjacency_list<>::edge_iterator> pe; property_map< adjacency_list<> , edge_index_t>::type inedge; inedge = get(edge_index_t(), processGraph); for (pe = edges(someGraph); pe.first != pe.second; ++pe.first) { cout << *pe.first << " "; cout << "index: " << inedge[*pe.first] << " \n"; } I can not figure out how to get an unique id for an edge and I can not understand how the property maps works from the documentation nor I can find how such a edge_index_t property is defined. Can somebody provide some help. With kind regards, Lodewijk __________________________________ Do you Yahoo!? Yahoo! Small Business $15K Web Design Giveaway http://promotions.yahoo.com/design_giveaway/

3 2

RE: [boost] Re: [program_options] Unicode support
by Ferdinand Prantl 06 Apr '04

06 Apr '04

Hi, > From: Vladimir Prus [mailto:ghost@cs.msu.su] > > > I understand, that stl support unicode for unicode is not the best, > > but there are facilities, that can provide required > functionality if > > properly extended/configured. > > Let's break the question in two parts. > > 1. Should 'unicode support' mean that there are two versions > of each interface, one using string and the other using > wstring? I think this kind of Unicode support is not good. It > means that each library which accepts or returns strings must > ultimately have double interface and be either entirely in > headers, or use instantinate two variants in sources -- which > doubles the size of the library. Not all libraries must have doubled interface or templated interface for any basic_string<C, T, A>. Different libraries have different string on their interface and one must convert among them. Typical is for example ANSI <-> UCS-2 on Windows. It is only convenient to be able to avoid such conversions when working with standard or widely used libraries, for example boost_regex, which provide templated interface and one can choose 8-bit or 16-bit character space (in future probly 32-bit with a new basic_string<> ;-). I think program_options, being also such a widely used and quite small library, shoud be also templated (no offence, meant just the size, not the importance ;-). > 2. Should program_options library use UTF-8 or wstring. As > I've said, neither is clear leader, but UTF-8 seems better. Here I disagree. Command-line shells work with all characters in the current locale (the whole 255 characters space of 8 bits is used). You would give the user a character array in UTF-8 encoding, which is not typical case today, one processes the parameters by basic_string<char>(argv[x]) in the current locale. I think you should simply use basic_string<> as a template and the encoding let on the caller providing its specialization or perform the conversion himself. Or support the encoding internally by providing an interface to set it, not to do it with a fixed encoding support, even if I like UTF-8 because it suppors full Unicode character range, not like UCS-2. Thinking more, you can expect rather short strings coming to program_options, not megabytes of text. For this usage is more suitable to use fixed-size character encodings because they are faster and easier to work with, having direct support in basic_string<>. Does anybody know, if there are plans to add something like basic_string<ucs2char_t> into the next C++ standard? > > I think, that there is no big reason to try to reinvent a wheel and > > provide all encopasing solution in the library like program_options. > > It should be enough if it will be unicode-enabled so it can > be used in > > the any specific scenario, provided that all necessary > facilities are > > on place. > > It's *far* from all encopassing solution. In fact, the > changes in program_options will include: > > 1. Adding ascii -> UTF-8 conversion in parsers 2. Adding > UTF-8 -> ascii conversion in value parsers 3. Adding unicode > parsers with UCS-4 -> UTF-8 conversion 4. Adding unicode > value parsers and UTF8 -> UCS-4 conversion > > That's all, and given that there's at least two UTF-8 codecs > announced on the mailing list, not a lof of work. And this > will add Unicode support without changing interface a bit. Yes, you are right; there is not much work to add the conversion code into the internals of program_options. I also wrote my own UCS-2 <-> UTF-8 encoding routines to use them for basic_string<char> <-> basic_string<wchar_t> conversion. However, I think, that we should reuse as much as possible and not to rewrite similar code in every library, which works with strings coming from a real user input. Your solution to support UTF-8 invisibly changes the interface anyway - not the text of prototypes directly but the behavior of the interface (encoding of strings). Nevertheless, you could support this encoding conversion not only by providing your own conversion routines, but rather by accepting existgin facets, which help streams similarly (as I wrote in the former e-mail). Then one could simply write a conversion facet once and use it for a stream input and also for a cmdline input, sharing the implementation. Ferda > - Volodya

2 1

Re: [boost] Re: Planning Next Release
by Aleksey Gurtovoy 06 Apr '04

06 Apr '04

Giovanni Bajo writes: > Aleksey Gurtovoy wrote: > > > I'd like to test the waters for the last week of June -- let's > > say Monday, June 28 -- being the target date for the next major > > release, which I am volunteering to manage. Correspondingly, > > June 21 or so would be a branch-for-release date, by which > > everything should be in CVS and in the releasable shape. > > I think we should try and allow a larger period of time to fix the > regression tests. It's definitely going to be more than one week, and the above wasn't meant to suggest otherwise (note "the releasable shape"). I did imply a different branch creation pattern than you suggest below, though. > I suggest we branch for instance 1 month before the release, and then > we restrict the branch only to the patches which stabilize the > regression tests (no new features allowed there). Plus, the trunk is > free for new features, and development is not stalled. Early branching would definitely give us more freedom for main trunk evolvement, and might even improve our release cycle in terms of predictability (although it's been my impression that in past almost all destabilizing checkins were due to "judgment errors" rather than accidental mistakes). On the other hand, any major development should be led in a separate branch anyway, so stalling is more a theoretical problem, IMO. The downside of branching early, is, of course, that the developers making fixes would have to do more work incorporating those both in the branch and the main trunk. I guess it's worth trying. What do others think? -- Aleksey Gurtovoy MetaCommunications Engineering

1 0