September 2011 - Boost - lists.boost.org

Re: [boost] [Review:Algorithms] Comments about search algorithms
by dpxguard-boost＠yahoo.gr 29 Sep '11

29 Sep '11

> Phil Endecott wrote: > > - A user faced with these three (or four) algorithms will need to > > choose one. The docs should offer some guidance, e.g. benchmarks or > > rules-of-thumb, about which to use. > In particular, I've not yet found a case where KMP is the best choice. > BM seems better in all the cases that I've tried and has the same or > better complexity bounds. (Well, it might perform slightly better when > the pattern length is 1 but in that case, std::find could be used.) > Does anyone know of any cases where KMP is likely to be better than > BM? (If we can't find any, is it worth including it?) I will try to give a complete answer... The most notable (and common) techniques for improving the sequence searching performance are the bad-character shift (occurrence shift) and the good-suffix shift (matching shift). The Boyer-Moore algorithm implements both these techniques, while the Boyer-Moore-Horspool only implements the bad-character shift and the KMP is merely based on the good-suffix shift. (Although the Boost.algorithm documentation only states that Horspool "is a refinement of the Boyer-Moore algorithm that trades space for time", this is by no means the most notable difference between these two algorithms.) In general, bad-character shift performs very good when the search-pattern is relatively small and the alphabet relatively large. On the other hand, good-suffix shift helps more when the pattern is relatively large and the alphabet small. In the most common cases of text searching, the patterns are usually small and the alphabets large, (> 255) hence the KMP (which is only using good-suffix shift) will almost never outperform BM in usual text-searching situations. (Note that KMP has not been used in the original std::search implementation of the STL, because even the straightforward (brute-force) algorithm has been considered to be superior in the average case!) The only practical case that I could think of, where the KMP could possibly outperform the BM are the DNA searches, where the patterns can be very long and the alphabet has only four characters. And even then, I would recommend careful performance testing, in order to ensure that KMP can be significantly faster than BM. However, in my opinion the most interesting and useful property of the KMP algorithm is not its performance, but its ability to help in building very generic sequence searching algorithms. In fact KMP has helped me to build an efficient generic sequence searching algorithm, which actually works with ranges accessed by single-pass (input) iterators. (See http://www.codeproject.com/KB/stl/single_pass_search.aspx and also in boost-vault/Algorithms) Needless to say that I would be more than happy to submit this work to boost, whenever there is an interest. Best regards Jim Xochellis Homepage: https://sites.google.com/site/xochellis/

3 7

[threads] Problem static linking with boost.thread during cross-compilation
by Victor Leschuk 29 Sep '11

29 Sep '11

Hello all, I have just met the following problem: I am cross-compiling boost 1.47 using $ x86_64-w64-mingw32-g++ --version x86_64-w64-mingw32-g++ (GCC) 4.5.4 20110822 (prerelease) On Linux box with the following command: ./b2 -a toolset=gcc target-os=windows threading=multi threadapi=pthread runtime-link=static --layout=versioned --build-type=complete install user-config.jam contains the following: using gcc : mingw : x86_64-w64-mingw32-g++ ; After that a linkage failed when building my project. I reproduced it with simple test program: #include <boost/thread.hpp> #include <iostream> using namespace std; struct test { void operator()() { cout << "Called" << endl; } }; int main() { test test_obj; boost::thread thr(test_obj); thr.join(); } The output looks like $ x86_64-w64-mingw32-g++ -Wall test_boost.cpp -I ./3rdparty/boost/include/boost-1_47 -L ./3rdparty/boost/lib -lboost_thread-mgw-mt-s-1_47 -o test_boost.exe In file included from ./3rdparty/boost/include/boost-1_47/boost/thread/win32/thread_data.hpp:12:0, from ./3rdparty/boost/include/boost-1_47/boost/thread/thread.hpp:15, from ./3rdparty/boost/include/boost-1_47/boost/thread.hpp:13, from test_boost.cpp:2: ./3rdparty/boost/include/boost-1_47/boost/thread/win32/thread_heap_alloc.hpp:59:40: warning: inline function ‘void* boost::detail::allocate_raw_heap_memory(unsigned int)’ declared as dllimport: attribute ignored ./3rdparty/boost/include/boost-1_47/boost/thread/win32/thread_heap_alloc.hpp:69:39: warning: inline function ‘void boost::detail::free_raw_heap_memory(void*)’ declared as dllimport: attribute ignored /tmp/ccSmsBnO.o:test_boost.cpp:(.text+0xde): undefined reference to `__imp__ZN5boost6thread4joinEv' /tmp/ccSmsBnO.o:test_boost.cpp:(.text+0xfb): undefined reference to `__imp__ZN5boost6threadD1Ev' /tmp/ccSmsBnO.o:test_boost.cpp:(.text+0x1c3): undefined reference to `__imp__ZN5boost6threadD1Ev' /tmp/ccSmsBnO.o:test_boost.cpp:(.text$_ZN5boost6threadC1I4testEET_NS_10disable_ifINS_14is_convertibleIRS3_NS_6detail13thread_move_tIS3_EEEEPNS0_5dummyEE4typeE[boost::thread::thread<test>(test, boost::disable_if<boost::is_convertible<test&, boost::detail::thread_move_t<test> >, boost::thread::dummy*>::type)]+0xd5): undefined reference to `__imp__ZN5boost6thread12start_threadEv' collect2: ld returned 1 exit status The same scenario works fine when linking with dynamic libboost_thread. Both static and dynamic builds work fine for Linux. What could be the cause of the problem and where to look? Does it worse posting a defect in bug-tracker? Thanks! -- Victor

2 3

Re: [boost] Gauging interest in patch submissions
by Hongli Lai 29 Sep '11

29 Sep '11

GMan wrote: > My only concern with this is in a case like this, for example: > > while (/* reading lines from file, parsing integers, otherwise ignoring*/) > { > try > { > myInts.push_back(boost::lexical_cast<int>(currentLineStr)); > } > catch (const boost::bad_lexical_cast&) // (has stack trace) > {} > } > > Will the stack trace make such an operation noticeably slower? I know it's > easy to retort with "don't use exceptions for flow control" but I believe my > concern is valid, even if it's a bit contrived of a situation. > > Perhaps we'd like to make it opt-in, like BOOST_THROW_EXCEPTION_TRACED or > something. I don't know what one considers "noticeably slower", but there will be overhead for sure. In our implementation, the creation of a tracable_exception object copies the current stack trace into itself. This may or may not be large depending on how many stack traces are defined. We've done our best to optimize our implementation but it's still an O(n) operation. There is one concern that I have with BOOST_THROW_EXCEPTION_TRACED. Will it affect the ABI? Suppose I compile against a Boost library that's installed on the system, and it was compiled without BOOST_THROW_EXCEPTION_TRACED. If I define BOOST_THROW_EXCEPTION_TRACED now will this change the exception hierarchy, and thus, the ABI? - Hongli -- Phusion | Ruby & Rails deployment, scaling and tuning solutions Web: http://www.phusion.nl/ E-mail: info(a)phusion.nl Chamber of commerce no: 08173483 (The Netherlands)

1 1

[any] new version
by Martin Bidlingmaier 28 Sep '11

28 Sep '11

Hi, I've written a new version of boost::any. It does not depend on rtti and uses a static integer value instead of std::type_info to store type information. My version passed the unit tests included in boost 1_47_0 on vc++2010 express and gcc 4.5.(mingw) . With this mail, I'm sending the source and a small quick-and-dirty benchmark. On gcc, moving and copying performance seems to be roughly the same, any_cast however is about 400% faster. On msvc++, moving and copying times rise by ca. 50%, while any_cast is about 50.000% faster (although I guess the speed boost measured in a real benchmark would be in the order of magnitude of the gcc speed up). I would appreciate if somebody measured the performance with proper tools and tested it on more platforms. Sincerely, Martin Bidlingmaier -- NEU: FreePhone - 0ct/min Handyspartarif mit Geld-zurück-Garantie! Jetzt informieren: http://www.gmx.net/de/go/freephone

26 87

[Review:Algorithms]
by Thorsten Ottosen 28 Sep '11

28 Sep '11

Hi Marshall, Some minor comments based on reading the docs only. A. xxx_of_val should be named xxx_of_equal or something IMO, or the name should simply be overloaded. B. Documentation for clamb: I dislike the "iff". Just say "if", because this doesn't clash with the case v==lo/hi. Use the names low/high or lower_bound/upper_bound. The "High point" terminology may be replaced with "upper bound" IMO. C. (ordered) I think the cannonical way is to use range_iterator<const R> instead of range_const_iterator<R> D. write #include <boost/algorithm/search.hpp> instead of Header 'search.hpp' (and similar for other names) in the documentation. E. A small code example on each page would be nice. F. Why is there no range-based versions of Boyer-Moore etc.? G. You could consider to present the iterator version and the range version of all algorithms on the same page. H. I couldn't fine any details about detail::BM_traits<>. Do I need to know about these? That's it. I think this is a useful, minor addition to Boost. I vote "yes" for inclusion. regards Thorsten

6 8

Re: [boost] [contract] syntax redesign
by Andrzej Krzemienski 27 Sep '11

27 Sep '11

> I think option B (current) is still better than option A because A requires > (1) more parenthesis and (2) the extra assertion word spelled out all the > times but A doesn't require the trailing `,` which is a plus. > Do you have a strong opinion about which one is better between A and B? I'd > say B is better but I'm quite biased at this point ;) Hi Lorenzo, Since you ask about personal preference, I think mine is option A. I can think of the following reasons: * Given the fairly complicated syntax for Boost.Concepts, option A appears to me to add more structure to the declaration, and makes parsing it easier. Especially that I imagine you could further simplify it: instead: var(auto old_size = CONTRACT_OLDOF size()) could be: var( old_size = CONTRACT_OLDOF size() ) * Since you put each assertion (or declaration) in a separate line anyway option B appears also elegant, but if someone starts putting multiple assertions in the same line, I think option A would be clearer. Compare: postcondition( x - 1 == y + 2, y - 1 == z + 2 ) postcondition( assert(x - 1 == y + 2) assert(y - 1 == z + 2) ) * If anyone writes a tool for generating documentation from Contracts, I think parsing would be easier with option A. * In option A, it is easier to tell assertions from declarations, I can easily count how many things are asserted * I suspect that you can use commas more freely inside assertion(), which reduces the necessity of using "identity" function workaround. But I could hardly call it a strong opinion. It is just my preference, and I will like Boost.Contract with either syntax. Regards, &rzej

1 0

[Review:Algorithms] Order of args to clamp
by Phil Endecott 27 Sep '11

27 Sep '11

Dear All, Some quick comments about the proposed clamp() function: template<typename V> V clamp ( V val, V lo, V hi ); I have a clamp function that orders the args low - middle - high, which seems like a more natural ordering to me. Is there some rationale or precedent for this middle - low - high ordering? I think the confusion matters in this case because type checking won't help to detect mistakes, and in the most obvious applications (e.g. sanitising input) it might be hard to spot an error. (In contrast, max(low,min(mid,high)) is unambiguous and not much more typing...) I note that this function takes its args by value, while std::min & max take const references. What is the rationale for this? Like min & max, clamp has a single type template parameter; what are the implications of this? For example, if I pass a mixture of std::string and const char* arguments, what will happen? Ideally, (a) all combinations would compile, and (b) conversions from const char* to std::string will only happen when necessary, not unconditionally for all of the arguments. (Maybe that is asking too much, though.) Regards, Phil.

17 91

[signals1/2] Alternative approach
by Sebastian Karlsson 27 Sep '11

27 Sep '11

A while back we evaluated signals 1 & 2 and came to the conclusion that the performance is to poor for it to be viable for our use. I have an idea which nets very close to the same invocation performance as std::vector< boost::function< > > ( an extra shared_ptr<> per ), with the caveat that ordering within a priority group is unspecified. Another downside compared to the implementation used now is that it needs extra bookkeeping per distinct priority group ( although from the docs for signals1 it would seem it does the same ). The gist of it is that we maintain a vector< > ( preferably we let the storage be parameterized ) in addition to a separate map for the priority groups. The map is keyed on the priority type, and the bookkeeping is: vector offset, group length & shift. Offset & length gives us the region in the vector. Shift tells us how much the group is shifted from its start offset since it was created. So each area is maintained as a sort of sliding window meets circular buffer. With this information we need touch no more than the either the last or first element ( depending on connect / disconnect ) of the groups with a lower priority than the one the element belongs to. Our vector contains a std::pair< function< signature >, shared_ptr< unsigned int > >, the unsigned int is used both as a cookie for the signals::connection to make sure that the connection is still valid, as well as a shared offset in regards to the shift of the group. The arithmetic is very light weight and shouldn't pose a bottleneck. Although I haven't implemented blocking & consistent disconnect / connect behavior during invocation yet I'm fairly certain they can be done in the following way: For blocking we can either use an additional bool which we branch on when invoking, or we can simply let the connection temporarily remove itself from the vector and insert itself once it's not blocking anymore, second solution probably preferred. Disconnect / connect during invocation is a trickier one. The way I'm thinking is to let the signal maintain the iterator as a member. When we disconnect / connect we can see if it equals end ( we're not invoking ), or if it doesn't then we know both the position as well as we're in the middle of an invocation. This way we can let the disconnect / connect logic modify the "cursor" in order to make sure that we get the desired behavior while not polluting our invocation loop. Pros: More cache friendly Very close to ideal case for invocation Cons: Using std::vector<> as a base naturally means we'll suffer from the usual growth caveats. However, if this matters the user could use a paged container with a decent growth strategy and come extremely close to the ideal case anyway. Thread safety within an invocation seems like it might be costly to implement, especially for vector<>. Slower connect / disconnect. All in all I think it's a big win for what I personally believe to be the most common usage of the signals library. I've attached a proof of concept along with a fairly primitive benchmark. There's a fragmented list version which can be enabled by a def, but it's naturally slow as hell to setup so it's disabled by default. The fragmented versions does not use exactly the same random sequence as the others, but hopefully it's somewhat fair. The vector variants somewhat "cheats" in that they reserve the space straight up, this is something which I personally feel should be exposed in the interface in someway anyway however. Here's some results: i5-2500k @ 3.3 ghz, decent mem etc: Using numConnections = 100000 numCalls = 100 numGroups = 200 rndSeed = 0 fragSpread = 10 Generating random sequence takes: 0.002 seconds Connecting to unfrag list:< function<> >: 0.008 seconds Connecting to vector< function<> >: 0.007 seconds Connecting to unfrag signals2< function<> >: 0.051 seconds Connecting to unfrag signals1< function<> >: 0.166 seconds Connecting to test::signals< function<> >: 1.38 seconds ... ... ... vector< function<> > invocation: 0.026 signals< function<> > unfragmented invocation: 1.632 signals2< function<> >unfragmented invocation: 1.404 list< function<> > unfragmented invocation: 0.03 test::signals< function<> > invocation: 0.023 signals< function<> > fragmented invocation: 9.97 signal2< function<> > fragmented invocation: 15.04 PS3 @ PPU ( Power PC ): Using numConnections = 100000 numCalls = 100 numGroups = 200 rndSeed = 0 fragSpread = 10 Generating random sequence takes: 0.001942 seconds Connecting to unfrag list:< function<> >: 0.060645 seconds Connecting to vector< function<> >: 0.034558 seconds Connecting to unfrag signals2< function<> >: 0.367024 seconds Connecting to unfrag signals1< function<> >: 2.02785 seconds Connecting to test::signals< function<> >: 10.0817 seconds ... ... ... vector< function<> > invocation: 0.695916 signals< function<> > unfragmented invocation: 5.91057 signals2< function<> >unfragmented invocation: 4.90188 list< function<> > unfragmented invocation: 1.03532 test::signals< function<> > invocation: 0.952999 signals< function<> > fragmented invocation: 33.4202 signal2< function<> > fragmented invocation: 47.0125

1 1

Rationale for shared_ptr/array including both ptr to control and ptr to data in class
by Olaf van der Spek 27 Sep '11

27 Sep '11

Hi, I was wondering why shared_ptr/array include both a ptr to the control block and a ptr to the data (ptr) in the class itself. I assume it's for performance reasons, but I didn't find anything about that in the docs. Greetings, Olaf

7 13

[Review:Algorithms] - Small review
by Christian Holmquist 26 Sep '11

26 Sep '11

Hi Marshall and others, I vote Yes to accept the library. Comments follows as I've encountered them in the documentation. - Description and rationale- "Boost.Algorithm is a collection of general purpose algorithms. While Boost contains many libraries of data structures, there is no single library for general purpose algorithms" boost/algorithm has existed for a while, but only the algorithm/string is properly documented (I think). Will boost/algorithm/minmax.hpp boost/algorithm/minmax_element.hpp be incorporated in the documentation? - Future plans- "My goal is to run regular algorithm reviews, similar to the Boost library review process, but with smaller chunks of code." Maybe a short summary on how a submission should work? Email subject tags etc, and what is required by the programmer (docs, code and tests). Since the library is at git (great!), maybe proposals could be done as push requests, or submitters share a link to their github repo or so. Having even a rough outline on the process would be better than none at all :) Do you intend to maintain the library at git? * I think the style linking to the algorithm sections from the main page is not optimal (i.e. Header: 'all.hpp' ). Maybe <Suggestion> Algorithms All Clamp Ordered Search </Suggestion> will read easier. - Reference section * Complexity is missing on each algorithm's page. * I would like if the page for each algorithm contains a small example like in the SGI's STL docs. They can be very trivial, but to me they are always reassuring that I've understood the docs correctly. Example assert(boost::clamp(0, 1, 2) == 1); * Every algorithm in the ordered section, has their iterator typename spelled FI. It would read easier as ForwardIterator. Dito for R -> Range. - General - Is it necessary to have both Range and free Iterator support? all.hpp use #pragma mark, probably not portable. (indeed, MSVC 2005 complains..) all.hpp could use std::begin() if available from std, unless there's a specific reason to always use the Boost version (?) search.hpp checks for a macro B_ALGO_DEBUG. It should read BOOST_ALGORITHM_DEBUG or similar to avoid name clashes. *** What is your evaluation of the design? Straight forward. Only worries a bit on the Range vs Iterator interfaces. It's probably impossible to satisfy everyone without providing both interfaces as has been done though. *** What is your evaluation of the implementation? Looks good to me. Well commented code with easy to read style. I can't comment on the correctness of the search.hpp algorithms though, but the tests are not passing here. *** What is your evaluation of the documentation? Everything needed is in the package, but the examples needs to be linked into the docs. *** What is your evaluation of the potential usefulness of the library? Immediately useful. good potential to evolve further. * How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? About 2.5 hours reading the docs, reading the code, running some examples and some of the tests. * Are you knowledgeable about the problem domain? Somewhat. *** Did you try to use the library? With what compiler? Did you have any problems? Visual Studio 2005 with native std (Dinkumware), Windows XP 64, 32 bit target, boost 1.46. All examples starting with int main ( int /*argc*/, char */*argv*/ [] ) { gives a warning on this compiler. search_example.cpp: d:\code\tests\tests\clamp.cpp(23) : warning C4138: '*/' found outside of comment int main ( int /*argc*/, char */*argv*/ [] ) { ^--------------------- add space here between * and / to make MSVC happy. d:\code\boost.algorithm\boost\algorithm\search.hpp(238) : warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data There are few similar warnings, the warning log is about 75 lines so would be good to get rid of. Running search_example.cpp in debug mode gives assertion failure in MSVC's Debug iterator code. I've not investigated further, all_example.cpp d:\code\tests\tests\clamp.cpp(23) : warning C4138: '*/' found outside of comment warning C4068: unknown pragma clamp_example.cpp error C2679: binary '<<' : no operator found which takes a right-hand operand of type 'std::string' (or there is no acceptable conversion) #include <string> fixes this search_test1.cpp Running 1 test case... Pattern is 8, haysstack is 37 chars long; unknown location(0): fatal error in "test_main_caller( argc, argv )": c:\program files (x86)\microsoft visual studio 8\v c\include\xstring(111) : Assertion failed: string iterator not dereferencable In Release mode, the output is Running 1 test case... Pattern is 8, haysstack is 37 chars long; unknown location(0): fatal error in "test_main_caller( argc, argv )": Invalid parameter detected by C runtime library .\clamp.cpp(71): last checkpoint *** 1 failure detected in test suite "Test Program" d:\code\tests\tests\clamp.cpp(71): last checkpoint *** 1 failure detected in test suite "Test Program" All tests pass except those related to search.hpp. I should point out that I just copied the source from each test into a toy visual studio project, but I've not had any troubles with that before. Sorry for a bit unorganized output of the tests, will take some time this evening to sort it out. Thanks to Marshall and Dave for the library and for managing the review - Christian

2 1