June 2008 - Boost - lists.boost.org

Inspection report and non-ASCII characters
by John Maddock 05 Jul '08

05 Jul '08

Would it be possible for the inspection report to print out the line containing the non-ASCII characters? There are a few files that are being flagged up, where I just can't find anything wrong with them :-( Thanks, John.

8 17

[dynamic_bitset] bug in to_ulong() in dynamic_bitset.hpp
by Boris Gubenko 05 Jul '08

05 Jul '08

A failure of dynamic_bitset library test dyn_bitset_unit_tests3 on Tru64/cxx and HP-UX/aCC revealed what appears to be a bug in <boost/dynamic_bitset/dynamic_bitset.hpp> introduced after Boost 1.34. The bug affects platforms where 'long' type is larger than 'int'. On both Tru64 and HP-UX, 'int' is a 32-bit type. On Tru64, 'long' is a 64-bit type. On HP-UX, 'long' is a 64-bit type in 64-bit data model (+DD64, which is how Boost testing is done on this platform) and 32-bit type otherwise. The test also fails on other platforms and while I cannot verify it, from the test output it apears that these platforms are affected by the same bug. What fails is the two testcases below in b.to_long() suite in boost/libs/dynamic_bitset/dyn_bitset_unit_tests3.cpp invoked with run_test_cases<unsigned int>() : { std::string str(ul_width - 1, '1'); boost::dynamic_bitset<Block> b(str); Tests::to_ulong(b); } { std::string ul_str(ul_width, '1'); boost::dynamic_bitset<Block> b(ul_str); Tests::to_ulong(b); } The code in <boost/libs/dynamic_bitset/bitset_test.hpp> converts dynamic_bitset<unsigned int> argument to unsigned long and checks that the appropriate bits in the result are set: result_type num = lhs.to_ulong(); // Be sure the number is right if (sz == 0) BOOST_CHECK(num == 0); else { for (std::size_t i = 0; i < sz; ++i) BOOST_CHECK(lhs[i] == (i < n ? nth_bit(num, i) : 0)); } This check fails for the bits in the upper half of the result. Here is an excerpt from dynamic_bitset<Block, Allocator>::to_ulong() function in <boost/dynamic_bitset/dynamic_bitset.hpp> in the trunk: result_type result = 0; for (size_type i = 0; i <= last_block; ++i) { assert((size_type)bits_per_block * i < (size_type)ulong_width); result |= (m_bits[i] << (bits_per_block * i)); } In our case, last_block = 1 and bits_per_block = 32. m_bits[i] has 'unsigned int' type. On the second loop iteration, a 32-bit integer is left-shifted by 32 bit positions. This triggers undefined behaviour. From 5.8 - Shift operators [expr.shift] : "The behavior is undefined if the right operand is [...] greater than or equal to the length in bits of the promoted left operand." With cxx and aCC, such shift results in zeroing of all bits of the left operand. Here is the code from <boost/dynamic_bitset/dynamic_bitset.hpp> in Boost 1.34 : unsigned long result = 0; for (size_type i = 0; i <= last_block; ++i) { assert((size_type)bits_per_block * i < (size_type)ulong_width); // gps unsigned long piece = m_bits[i]; result |= (piece << (bits_per_block * i)); } Here on the second loop iteration, a 64-bit long is left-shifted by 32 bit positions which yields the intended result. x.cpp below illustrates the difference between trunk and Boost 1.34: x.cpp ----- #include <stdio.h> #include <limits.h> int main() { unsigned int m_bits[] = { UINT_MAX, UINT_MAX }; unsigned int bits_per_block = sizeof(int) * CHAR_BIT; unsigned long result = 0; for(int i = 0; i <= 1 ; ++i) { #ifndef Boost_1_34 result |= (m_bits[i] << (bits_per_block * i)); #else unsigned long piece = m_bits[i]; result |= (piece << (bits_per_block * i)); #endif printf("result = %llx\n", result); } } cxxosf.zko.hp.com> cxx x.cpp && a.out result = ffffffff result = ffffffff cxxosf.zko.hp.com> cxx x.cpp -DBoost_1_34 && a.out result = ffffffff result = ffffffffffffffff cxxosf.zko.hp.com> Replacing result |= (m_bits[i] << (bits_per_block * i)); with unsigned long piece = m_bits[i]; result |= (piece << (bits_per_block * i)); on the trunk makes the test succeed: verified on Tru64/cxx and on HP-UX/aCC. In <http://lists.boost.org/Archives/boost/2007/12/131126.php> I said: > It looks like a regression in boost/dynamic_bitset/dynamic_bitset.hpp > introduced in 1.35, but even in 1.34.1, it seems to me that the code in > to_ulong() is relying on undefined behaviour. What I was referring to is the code in 1.34 applying bitwise shift operator to a long operand. It would trigger an undefined behavior in C where "The operands shall have integer type", but it is legal in C++ where "The operands shall be of integral or enumeration type". So, I no longer claim that the code in 1.34 is relying on undefined behaviour. And the last thing: why the bug affects only platforms where 'long' type is larger than 'int'? Because if they are the same, as is the case on VMS or HP-UX in 32-bit data model, last_block = 0 and the single iteration of the for loop just assigns m_bits[0] to the result: the right operand of the shift operator is zero. To fix the problem, I suggest to replace the buggy code in <boost/dynamic_bitset/dynamic_bitset.hpp> in the trunk with that in 1.34. If there are no objections, I can do it. Thanks, Boris

3 3

[math] pow redux and observations
by John Moeller 05 Jul '08

05 Jul '08

I noticed that the trunk now has corrections for pow<N> to reduce template instantiations. However, the "N=1" specialization isn't correct. The second template parameter (bool odd) should be "true," not "false." I could be mistaken that 1 is an odd number, though. :-) As it is now, the whole template would work, but would ignore the 1/false specialization and create a new 1/true specialization that just multiplies "base" by the result of 0/false. Changing 1/false to 1/true will just cause it to return "base," as desired. Note also, please, that positive_power<0,false> returns 1 when base is 0. Since 0^0 is technically undefined, the "N=0" case should probably be handled at a higher level (in power_if_positive, perhaps), and cause an error, just like negative powers of 0. If that is done, the 0/false case could be removed from positive_power.

9 50

libs/dynamic_bitset/bitset_test unmerged changes?
by David Abrahams 05 Jul '08

05 Jul '08

Should these be in the trunk? Index: bitset_test.hpp =================================================================== --- bitset_test.hpp (revision 40672) +++ bitset_test.hpp (working copy) @@ -1006,7 +1006,8 @@ bool did_throw = false; try { - s << b; + static_cast<void> + (s << b); } #if defined BOOST_OLD_IOSTREAMS catch(...) { @@ -1081,7 +1082,8 @@ const std::ios::iostate except = is.exceptions(); bool has_stream_exceptions = true; try { - is >> b; + static_cast<void> + (is >> b); } catch(const std::ios::failure &) { did_throw = true; -- Dave Abrahams Boost Consulting http://www.boost-consulting.com

2 1

[allocators] Article and library about allocator improvements
by Ion Gaztañaga 03 Jul '08

03 Jul '08

Hi, While developing of Interprocess, I started to think about optimizations for STL-like allocators. I even wrote a proposal about some new features for standard containers (N2045, http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2045.html) that was not received with much interest by the committee. I refined those improvements for shared memory containers and I measured performance benefits when containers take advantage of more advanced allocators. The last few months, I've taken another step forward and I've developed a small Boost library based on a modified DLmalloc code. I've developed some allocators above the modified DLmalloc and I've measured the performance benefits that can be obtained with optimized allocators that offer some new features like buffer expansion, burst allocation, etc... Test were performed using Interprocess containers, but instead of shared memory allocators, I've compared DLmalloc based heap allocators against the standard std::allocator<>. I've written a long article with my explanations and conclusions: http://www.drivehq.com/web/igaztanaga/allocplus/ The library (includes the article) can be downloaded here: http://www.drivehq.com/web/igaztanaga/allocplus.zip The library has no docs (the article explains all the basics) and does not surely meet Boost standards. The only goal of this article, the library and this post is to share information with other Boost container authors and discuss possible improvements that maybe could be applied to Boost containers. If those improvements are considered interesting, it would be great if we could write a library with allocators and develop some "existing practice" with boost containers before writing a future proposal to improve allocators. I plan to publish the article somewhere (supposing someone is crazy enough to publish it ;-) ) but first I wanted to share this with boosters to try to improve the article/library and know if developers are interested in allocator issues. Comments are welcome! Regards, Ion

5 11

A branching/tagging idea for releases
by Daryle Walker 03 Jul '08

03 Jul '08

Here's an idea on how we can manage our source-control for releases: 1. When we're ready to start a new release, make a branch from the trunk with a name like release_1_??_x (e.g. "release_1_37_x") 2. Finalize what needs to be done to make that branch ship-worthy 3. Make a tag from that branch with a name like release_1_??_0 (e.g. "release_1_37_0") 4. Export that tag to a fresh directory and create the appropriate archive file to upload to SourceForge, etc. (The directory shouldn't have any source-control debris in it. Maybe this needs to be done once on each platform to ensure the best archive format and the proper line-ending markers.) 5. If bug-fixes for that release need to be done, continue to use the release_1_??_x branch (and merge the fixes back to the trunk, or vice versa) 6. Make a new tag named like release_1_??_1 and ship it as in [4] for any bug-fix releases (increase the number for subsequent bug- fixes published) 7. New stuff and substantial changes go on the trunk; publish those by going back to [1] with a release_1_(??+1)_x branch -- Daryle Walker Mac, Internet, and Video Game Junkie darylew AT hotmail DOT com

6 14

Documentation for Boost libraries
by Edward Diener 03 Jul '08

03 Jul '08

In general Boost has many libraries with good to absolutely first class documentation. Among the many Boost libraries I have used, all with good documentation, are regex, iostreams, function, bind, optional, signals. tribool, tokenizer, and spirit, but of course there are many other libraries with great documentation in Boost. One of my prerequisites for a library with good documentation is that some sort of overview must be given for the functionality of the library so that the end user knows what is available, and what each available part of the library generally entails. For me this overview must be written out and not just given as a series of programming examples, whether in the form of tutorial code, example code, or whatever one wants to call explanations of functionality based almost purely on some hypothetical program or mini-program. In this I may be different from many programmers but I just find it very hard to understand library functionality from example code without basic generalized explanations of functionality. Today, looking for a socket library to use in my C++ code I decided to look at the documentation for ASIO. I was shocked to find out that there is no overview of the functionality of this library at all, but just a series of tutorials and examples and then a reference listing a multitude of classes. I do not believe this is adequate for explaining library functionality and puts a totally unnecessary burden of the end-user when learning the functionality of a library. I would like to encourage Boost developers to always give an overview of the functionality of their library and its general concepts and classes, as almost all libraries currently do. While I am appreciative of tutorials and exmaples as an extra learning device, I do not believe that library documentation should exist with some sort of overview, as mentioned above. I have written this because I am disturbed that ASIO was accepted with the sort of documentation which has been created for it. I do not think such a way of presenting a library is adequate for an organization like Boost whose overall excellence is acknowledged throughout the C++ community.

6 12

Is "+" safe for file names?
by Rene Rivera 02 Jul '08

02 Jul '08

Just noticed this directory name: libs/filesystem/test/vc++.boost.filesystem Is "+" safe to use across file systems? Hm, actually I'm fairly sure more than one "." is not safe. For example in the MVS systems? -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org (msn) - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail

7 11

[asio] failure in ~wrapped_handler
by Dean Michael C. Berris 02 Jul '08

02 Jul '08

Hi everyone, I recently ran into a problem with Boost.Asio while using a slightly modified version of the HTTP 2 example server. The debugger trace (attached) shows where a segfault had occurred in the code. The short of it is that when ~wrapped_handler is called, the call to boost::detail::atomic_exchange_and_add fails when boost::detail::sp_counted_base::release is invoked. It occurs when a handler_wrapper is deleted/destructed (~handler_wrapper is called) in the strand_service. I'm just wondering what could be causing this -- it's worth noting that under not-so-heavy load, the problem does not manifest itself; but soon enough when the load on the server goes up (more incoming connections) the server crashes consistently. Any pointers where I should be looking? I'm using Boost.Trunk revision number 43296. -- Dean Michael Berris Software Engineer, Friendster, Inc.

3 2

Anyone interested in a generic loop (and loop unrolling) library ?
by Hui Li 01 Jul '08

01 Jul '08

Loops, in particular loop-unrolling, can be made generic, easy to write, and accessible to everybody. With the help of Boost.Lambda (and a few other boost libraries), we can easily construct arbitrary loops that are unrolled at compile-time. I haven't implemented the full library yet, in fact I've implemented only a few very preliminary functionalities (hardly flawless), it serves only to illustrate the ideas. From what they can do, a possible Boost.Loop library looks very promising. However I feel that I need much help on the design of such a library. The implementation could be straightforward (with the help of so many good libraries in Boost), once a good design is found. Hopefully there are interested people here so I won't have to work on my own. (If there already exists such a library, please let me know.) In the example, you can see how easy it is to construct arbitrary (limited only by Boost.Lamda) loops that are generically unrolled at compile time. With an optimizing compiler, it can be as efficient as hand-written codes. Not only that, but also can we make (almost) any linear-algebra libraries work together without difficulty. In the example, we don't need any extra coding to make a c_array, std::vector, or even a column_view of a ublas::matrix to work together. In the following I give the full example code to illustrate what it's capable of and how to use it. The header file, loop.hpp, is attached at the end of this post. The example has been tested with apple-gcc4.0.1 . Thanks, Hui /**********************************************************************************************************/ #include <vector> #include <iostream> #include <iterator> #include <functional> #include <boost/array.hpp> #include <boost/numeric/ublas/matrix.hpp> #include <boost/numeric/ublas/matrix_proxy.hpp> #include <boost/numeric/ublas/io.hpp> #include "loop.hpp" using namespace std; using namespace boost; using namespace boost::lambda; using namespace boost::numeric; using namespace boost::numeric::ublas; using namespace loop; int idt(int i) { return i; } int main(int argc, char* argv[]) { const size_t N = 8, M = 3; boost::array<double,N> v; std::vector<double> w(N); matrix<double> m(N,M); // All loops are unrolled at compile time // indtrans acts like an array whose individual element equals its index index_transform<double(int)> indtrans(&idt); // equivalent to for(i=0;i<N;i+=2){ v[i]=w[i]=1; } loop_over< linear_range<0,N,2> >( _1 = _2 = 1, v, w ); // equivalent to for(i=1;i<N;i+=2){ v[i]=i; } loop_over< linear_range<1,N,2> >( _1 = _2, v, indtrans ); // equivalent to for(i=1;i<N;i+=2){ w[i]=i; } loop_over< linear_range<1,N,2> >( _1 = _2, w, indtrans ); // loop_over< linear_range<1,N,2> >( _1 = _2 = _3, v, w, indtrans ); // I wonder why this line doesn't compile with gcc4?? m.clear(); // set m to a zero matrix // equivalent to for(i=1;i<N;++i){ column(m,2)[i]=1+i*2; } loop_over< linear_range<0,N,1> >( _1 = 1+_2*2, loop::ref(column(m,2)), indtrans ); // equivalent to for(i=1;i<2;++i){ v[i]*=-2; } followed by for(i=N-3;i<N;++i){ v[i]*=-2; } loop_over< link_range<linear_range<0,2,1>,linear_range<N-3,N,1> > >( _1 *= -2, v); // vertically print out v and w, side by side loop_over< linear_range<0,N,1> >( cout << _1 << constant('\t') << _2 << constant('\n'), v, w ); cout<<endl; // print out m cout << m << endl << endl; double s = 0; // calculate the inner-product of w and column(m,2) (when the column is viewed as a vector/array) loop_over< linear_range<0,N,1> >( _1 += _2*_3, loop::ref(loop::scalar(s)), w, loop::ref(column(m,2)) ); cout << s << endl << endl; return 0; } /**********************************************************************************************************/ Here is the loop.hpp /********************************************************************************************************** * loop.hpp * Created by Hui Li on 6/26/08. ******************************************************************************************************** */ #ifndef LOOP_H #define LOOP_H #include <boost/static_assert.hpp> #include <boost/ref.hpp> #include <boost/lambda/lambda.hpp> #include <boost/function.hpp> #include <boost/function_types/function_arity.hpp> #include <boost/mpl/if.hpp> #include <boost/mpl/bool.hpp> #include <boost/mpl/assert.hpp> namespace loop { // the empty range struct empty_range {}; // a range starting from Start, ending at End, with stride Stride template < int Start, int End, int Stride = 1 > struct linear_range { BOOST_MPL_ASSERT(( boost::mpl::bool_<(Start*Stride<End*Stride)> )); static const int first = Start; typedef typename boost::mpl::if_ < boost::mpl::bool_< ((Start+Stride)*Stride < End*Stride) >, linear_range < Start+Stride, End, Stride >, empty_range >::type next_range; }; // link two arbitrary ranges template < typename LeftRange, typename RightRange > struct link_range { static const int first = LeftRange::first; typedef typename boost::mpl::if_ < typename boost::is_same < typename LeftRange::next_range, empty_range >::type, RightRange, link_range < typename LeftRange::next_range, RightRange > >::type next_range; }; // implementation of the unrolled loop over a range template < typename LambdaExpr, typename Range > struct loop_Impl { template < typename V1 > void operator()(LambdaExpr& expr, V1& v1) { expr(v1[Range::first]); loop_Impl < LambdaExpr, typename Range::next_range >()(expr,v1); } template < typename V1, typename V2 > void operator()(LambdaExpr& expr, V1& v1, V2& v2) { expr(v1[Range::first],v2[Range::first]); loop_Impl < LambdaExpr, typename Range::next_range >()(expr,v1,v2); } template < typename V1, typename V2, typename V3 > void operator()(LambdaExpr& expr, V1& v1, V2& v2, V3& v3) { expr(v1[Range::first],v2[Range::first],v3[Range::first]); loop_Impl < LambdaExpr, typename Range::next_range >()(expr,v1,v2,v3); } }; // looping over an empty range does nothing template < typename LambdaExpr > struct loop_Impl < LambdaExpr, empty_range > { template < typename V1 > void operator()(LambdaExpr&, V1){} template < typename V1, typename V2 > void operator()(LambdaExpr&, V1&, V2&){} template < typename V1, typename V2, typename V3 > void operator()(LambdaExpr&, V1&, V2&, V3&){} }; // helper functions template < typename Range, typename LambdaExpr, typename V1 > void loop_over(LambdaExpr& expr, V1& v1) { loop_Impl<LambdaExpr,Range>()(expr,v1); } template < typename Range, typename LambdaExpr, typename V1, typename V2 > void loop_over(LambdaExpr& expr, V1& v1, V2& v2) { loop_Impl<LambdaExpr,Range>()(expr,v1,v2); } template < typename Range, typename LambdaExpr, typename V1, typename V2, typename V3 > void loop_over(LambdaExpr& expr, V1& v1, V2& v2, V3& v3) { loop_Impl<LambdaExpr,Range>()(expr,v1,v2,v3); } // map a call to operator[] to a unary function all template < typename Function > struct index_transform { BOOST_STATIC_ASSERT(( boost::function_types::function_arity<Function>::value == 1 )); typedef boost::function<Function> func_type; typename func_type::result_type operator[](typename func_type::argument_type x) { return func(x); } template < typename FuncTp > index_transform(FuncTp f):func(f){} private: func_type func; }; // force removal of const-ness of a reference // not so elegant but should be okay template < typename T > T& ref(const T& t) { return const_cast<T&>(t); } // independent of whatever the index might be, it always returns the reference to the same variable template < typename T > struct scalar_Impl { T& operator[](int) { return ref; } scalar_Impl(T& t):ref(t){} private: T& ref; }; // scalar helper template < typename T > scalar_Impl<T> scalar(T& t) { return scalar_Impl<T>(t); } } // namespace loop #endif /**********************************************************************************************************/

4 3