February 2012 - Boost - lists.boost.org

[hash] regular behaviour of hash function for double values
by Kai Schroeder 07 Feb '12

07 Feb '12

Hi, the following code: double fStart = 31337; double fEnd = 65536; double fStep = fStart - fEnd; fStep /= 512; for (int i = 0; i < 512; i++) { std::cout << std::hex << boost::hash<double>()(fStart + i * fStep) << std::endl; } leads to the output of surprisingly regular hash values (see end of mail). This seems to occur mainly if a power of two number of points is sampled equidistantly within an interval. Especially since some hash table implementations (e.g. intel's threading building blocks) only use the last few bits to determine the bucket, this can lead to very bad caching behaviour for the rather common case of regularly sampled points (equidistant samplings of power of 2 are e.g. common in computer graphics and numerical algorithms). Are you aware of this? Would you consider it a bug? Best regards, Kai Schroeder Output of code using Boost 1.48 (similar behaviour can be observed for other start and end values and other power of 2 step counts): ff9f20000000000f e5042f900000000f 52f13f200000000f 297ac8b00000000f 48721e400000000f 2f9b8dd00000000f 3df0b1600000000f d19cf2f00000000f 9201c800000000f 9409aa100000000f 7cb3fba00000000f d7a42b300000000f b15982c00000000f 5f7900500000000f e0ba85e00000000f 255ee5700000000f cec119000000000f 955bd6900000000f 6a58f4200000000f 843a37b00000000f 590797400000000f 7e8db4d00000000f 1721b6600000000f 37acf9f00000000f df5f65800000000f 88b1a5100000000f 494a0a00000000f bb1082300000000f 1ea62bc00000000f ...

11 43

unordered_map 32bit VS 64bit
by Avikam Bara-nes 07 Feb '12

07 Feb '12

Hi, I'm porting a project from Linux 32bit to Linux 64bit. While doing so I notice that I have a dramatic performance degradation with unordered_map. My platform is Ubuntu 10.4.3 64bit running on Intel Xeon X3220 but, I tested similar program on WindowsXP SP2 64bit which running on Intel i5 and I get the same degradation. My boost version is 1_46_1. I use the following program in order to test unordered_map - Hop I didn't make any copy mistakes :-). #include <stdio.h> #include <sys/time.h> #include <boost/unordered_map.hpp> int main(int argc, char *argv[]) { struct timeval tv_before, tv_after; boost::unordered_map<boost::int_32_t, boost::int32_t> boost_map; boost_map[1] = 2; boost_map[3] = 4; gettimeofday(&tv_before, NULL); for(int i = ; i < 100000; i++) boost_map.find(1); gettimeofday(&tv_after, NULL); printf("Boost time is %f \n", (double)(((tv_after.tv_sec - tv_before.tv_sec) * 1000000 + (double)(tv_after.tv_usec - tv_before.tv_usec)) / 100000)); return 0; } I use the following compile commands: 64bit - g++ -O3 perf_test -o perf_test_64bit 32bit - g++ -m32 -O3 perf_test -o perf_test_32bit When I run the 32bit program I get 0.5 sec When I run the 64bit program I get 1.54 sec Any ideas ???

6 16

[MPI]: All tests failing -- branch and trunk
by Steve M. Robbins 07 Feb '12

07 Feb '12

Hi, I have been running a daily test of the Boost release branch for a long time. It normally takes an hour or so. However, two days ago (rev 76868), the build time shot up to over 9 hours. The same thing happened for testing the Boost trunk branch. Others on the boost-test list have experienced the same and Jurgen Hunold pointed out that all the MPI tests are failing. According to http://www.boost.org/development/tests/trunk/developer/mpi.html this is true for about half the tester machines. Should probably be fixed before 1.49 is released! -Steve

1 0

[functional] adding overload
by Lorenzo Caminiti 06 Feb '12

06 Feb '12

Hello all, During Boost.Local review, it was proposed to move the overload template out of Local and in Boost.Functional. What would the best name for it? 1) boost::overload_function (my preference) 2) boost::overloaded_function 3) boost::function_overload 4) boost::functional::overload (Boost.Functional seems to use boost:: directly and not boost::functional-- to mimic C++11 <functional>). For example: void print_text(char const* text) { std::cout << text << std::endl; } void print_add(int x, int y = 10) { std::cout << (x + y) << std::endl; } boost::function<void (int, int)> f = print_add; boost::overload_function< void (char const*) , void (int, int) , void (int) > print(print_text, f, print_add); print("abc"); // abc print(1, 2); // 3 print(1); // 11 Docs: https://svn.boost.org/svn/boost/sandbox/local/libs/local/doc/html/boost/loc… Thanks. --Lorenzo

8 18

[filesystem] windows/posix inconsistencies.
by Thorsten Ottosen 06 Feb '12

06 Feb '12

Hi Beman, I noticed the following in v3 whie hunting down a bug in my unit test: # ifdef BOOST_WINDOWS_API const std::string string() const { return string(codecvt()); } const std::string string(const codecvt_type& cvt) const; // string_type is std::wstring, so there is no conversion const std::wstring& wstring() const { return m_pathname; } const std::wstring& wstring(const codecvt_type&) const { return m_pathname; } # else // BOOST_POSIX_API // string_type is std::string, so there is no conversion const std::string& string() const { return m_pathname; } const std::string& string(const codecvt_type&) const; const std::wstring wstring() const { return wstring(codecvt()); } const std::wstring wstring(const codecvt_type& cvt) const # endif I can understand, that it is more efficient to return a reference when a referenec can be formed. However, consider const std::string& ext = iter->path().extension().string(); This is fine on windows, as a temporary is returned, and its lifetime is extended; on linux, extension() returns a temp path object, from where we get a reference. I'm not a 100% sure, but I think the C++ standard does not guarantee that this path object be kept alive. The net efffect is a crash on linux. I'm not sure its a great idea to have different reference-ness on the return types here. It could lead to other subtle differences. kind regards -Thorsten

9 14

[mpl] Nested Scopes w/Placeholders
by Kitten, Nicholas 06 Feb '12

06 Feb '12

I've recently been doing a lot mpl::folding (in boost 1.47), and I was disappointed when I realized that a nested fold wouldn't work with a lambda expression passed to the inner call, as in this example which extracts elements of sequences: using namespace mpl; using namespace mpl::placeholders; typedef vector< vector<char>, vector<int> > vec_of_vecs; typedef vector< char, int > expected_result; typedef fold< vec_of_vecs, vector<>, fold< _2, _1, *push_back< _1, _2 >* // wrong - refers to outer arguments > >::type result; BOOST_MPL_ASSERT(( equal<result, expected_result> )); After looking around for a solution I came across an old thread<http://lists.boost.org/boost-users/2004/12/9269.php> on the same subject, where it was suggested using mpl::protect might work. Indeed, the following modification does what I want, without any changes to the library: typedef fold< vec_of_vecs, vector<>, fold< _2, _1, *protect< lambda< push_back< _1, _2 > >::type >* // correct > >::type result; Of course, this is a little on the verbose side. In the old thread, Daniel Wallin suggested suggested changing the definition of lambda to yield syntax more in line with the runtime version of boost::bind, but that would apparently break other code. What about instead adding a specialization to protect for placeholder expressions? Indeed, the source in protect.hpp seems to indicate such a thing might've been in the works at one point, but I didn't see any more references to it: template< typename BOOST_MPL_AUX_NA_PARAM(T) * , int not_le_ = 0 // not lambda expression?* > struct protect : T { ... typedef protect type; }; What about changing it to something like this? template< typename BOOST_MPL_AUX_NA_PARAM(T) * , typename not_le_ = typename is_lambda_expression< T >::type* > struct protect : T { ... typedef protect type; }; *template< typename T >* *struct protect< T, mpl::true_ > : lambda<T>* *{* *...* *typedef protect type;* *};* Doing so would remove the inner lambda<...>::type expression, yielding the expected syntax. Could that be done without messing up everything else? Thanks, *Nick Kitten* Software Engineer Center for Video Understanding Excellence ObjectVideo, Inc.

3 6

[gsoc] Summer of Code 2012
by Andrew Sutton 05 Feb '12

05 Feb '12

Has it been a year already? Google has just announced the 2012 Summer of Code program, and we'd like to participate again. I think last year was particularly successful. We reviewed and accepted two new libraries and moved several more into a state where they can be reviewed. This year should be interesting. We have a new C++ programming language, several libraries in the standard, and we want more. Many, many more. If you didn't get a chance to watch Herb Sutter's keynote talk at GoingNative 2012, you should. It will be available on-demand from this site some time in the next few days. http://channel9.msdn.com/Events/GoingNative/GoingNative-2012 [Spoiler alert] Herb wants libraries -- and not just tools for programmers. C++ wants UI libraries, graphics, database connectivity, cryptography, XML, network protocols, SMS support, back office support, etc. I would love to see some of these libraries developed through Boost. In C++11. Bjarne Stroustrup said he hadn't seen this much energy around C++ in a decade. People are excited to be writing C++11 -- and they're younger people for the first time in a long time. We should be capitalizing on this. If you have library ideas, please email me here: andrew.n.sutton -at- gmail.com. I will collect these and put them on the idea page. Some comments for mentors: I'm not concerned with summer-long time constraints this year. I'd really like to establish long-term relationships with our students. I'm not strictly concerned with backwards compatibility support. I think we should be encouraging students (and people in general) to write in C++. Hiding new language features behind compatibility macros isn't exciting. It might be fine for when time comes for releases, but let's not require students to build release-ready software. This year, I'd like to see proposals for *new* libraries to be written in C++11. Andrew

1 0

[bgl] Maximum Adjacency Search
by Fernando Vilas 04 Feb '12

04 Feb '12

Hi, Well, it took me longer than "a few days" to make time to clean up the code, but here is a patch against today's SVN-trunk. It will probably need more cleanup before it is ready, but it should be at a point where public review will help. -- Thanks, Fernando Vilas fvilas(a)iname.com

2 1

1.49.0 Beta 1 Release Candidate
by Steve M. Robbins 04 Feb '12

04 Feb '12

Hi, I'm starting a build (using the Debian package configuration) and noticed several known bugs still present: These two bugs are patched on the trunk but not release. Can these be moved over, please? https://svn.boost.org/trac/boost/ticket/4657 https://svn.boost.org/trac/boost/ticket/6338 This issue has a simple patch that works around the problem for gcc versions prior to 4.6.3: https://svn.boost.org/trac/boost/ticket/5538 This issue has a patch that works (is used in Debian), though a jam expert could likely find a better fix: https://svn.boost.org/trac/boost/ticket/6286 Regards, -Steve (Boost maintainer for Debian)

5 9

[random] Determining interest in Pseudo-Random Functions and Counter Based Random Number Generators
by John Salmon 03 Feb '12

03 Feb '12

I am co-author of "Parallel Random Numbers, as Easy as 1, 2, 3". which recently won the best paper award at the IEEE/ACM SC'11 conference. I'm also co-author of the related Random123 library: http://deshawresearch.com/resources_random123.html My colleagues and I would like to contribute a similar library to Boost as well. This isn't just a "port" of the Random123 library. Boost's focus on C++ allows for both simplification and clarification of the ideas in the paper and the original library. I've done a preliminary draft of some code that fits nicely into the existing Boost Random library. It's incomplete, but I think there's enough for folks to look at and gauge further interest. A zip file with a boost-like layout is here: http://thesalmons.org/john/random123/prf_boost/prf_boost.zip The README from the zip file is appended. I look forward to your comments and suggestions. John Salmon -------------- README ------------------- This extension of the Boost Random library adds "Pseudo-Random Functions" (PRFs) as an alternative to conventional "Uniform Random Number Generators" and "Random Number Engines". It is an incomplete draft, intended to start a conversation. It is not ready-to-use library. The case for using PRFs in Monte Carlo and statistical simulation is given in the paper, "Parallel Random Numbers -- As Easy as 1, 2, 3", by Salmon, Moraes, Dror & Shaw, in the Proceedings of SC'11: http://dl.acm.org/citation.cfm?doid=2063405 also available at http://www.thesalmons.org/john/random123/papers/random123sc11.pdf PRFs have been known for many years, but either their advantages were unrecognized or they were thought to have poor performance with respect to conventional URNGs. The paper shows that PRFs exist with excellent statistical properties, and highly desirable space and time characteristics. Also, note that the terminology in the paper, is slightly different from that used here. In the paper they're called "Counter Based Random Number Generators". PRFs are well-known in cryptography, but the PRFs here (except for AES) are *not* cryptographically strong. Nevertheless, they have satisfied extensive tests of statistical quality and should satisfy the needs of Monte Carlo simulation and other fields that routinely use random numbers. PRFs are pure functions (functors in C++) with the property that they return "random" values, even when presented with highly regular inputs (see the paper for a discussion of how randomness is empirically established). The user of a PRF assembles the input, calls the function and obtains a "random" result which can then be used in the same way as a value obtained from a conventional URNG or Engine. In contrast, a conventional Random Number Engine has a much more limited API: the caller is restricted to seeding the engine, once and for all, and then repeatedly asking for the "next" random value. PRFs eliminate the sequential dependence inherent in conventional RNGs. With a PRF, one can compute the "n-th" value in O(1) time, without stepping through earlier values. In fact, with a PRF, the notion of an "n-th" value loses its significance. One simply obtains a random value. Eliminating sequential dependence can be especially useful for parallel, multi-threaded applications. For example, one can assemble a PRF's input from application-level data, e.g., object id, voxel number, user-supplied seed value, timestep, iteration number, etc. This kind of application data is unchanged when the application is parallelized, and is generally invariant as more or fewer threads become available. The resulting random value is thus invariant to the parallelization as well. This can be enormously helpful when correctness dictates that separate "parallel" threads should use the same "random" value, or when trying to reproduce results on different platforms with different levels of parallelism. Four PRFs are implemented: threfry, philox, ars and aes, each implemented in its own header file, e.g., boost/random/ars.hpp. The file boost/random/counter_based_urng.hpp implements the counter_based_urng adapter class, which creates a Uniform Random Number Generator from a PRF, allowing programmers to use the PRF APIs together with Boost or C++1x Random Number Distributions. The file boost/random/counter_based_engine.hpp implements the counter_based_engine adapter class, which creates a Uniform Random Number Engine from a PRF, allowing PRFs to be used with software that relies on the Uniform Random Engine concept. For better or worse, wrapping a PRF in a counter_based_engine adapter completely obscures the PRF's API and reintroduces sequential dependence. The zip file also contains a copy of the candidate Boost Endian library copied from boost sandbox. Endian-swapping is required by the AES engine on bigendian machines, and the library meets our needs. Concepts -------- All PRFs have a domain_type and a range_type and with d a "permissible" object of the domain type, the expression: prf(d) is of the range_type. In many cases, every possible value of domain_type will be "permissible", but we leave open the possibility that the prf may place restrictions on the values of domain_type that produce "random" results. The values returned by the prf should be "random", a concept that is difficult to make both precise and achievable. Roughly, for most sequences, including highly "regular" sequences, of distinct, permissible domain_type values, d1, d2, d3, ..., the sequence: prf(d1), prf(d2), prf(d3), ... is statistically indistinguishable from a sequence of independent, uniformly distributed random variables sampling the range_type. In most cases, the output of the prf will uniformly sample all possible values of the range_type, but we leave open the possibility that it may only sample a subset and that the output is only uniform over the subset. Keyable PRFs have a key_type, a corresponding constructor, and setkey() and getkey() member functions. Keyable PRFs constructed with different keys must also be statistically independent. I.e., even for highly regular sequences of distinct (key, domain) pairs: (k1, d1), (k2,d2), ... the sequence: Prf(k1)(d1), Prf(k2)(d2), ... is statistically indistinguishable from a sequence of independent random uniformly distributed samples of the range_type Keyable PRFs may also be Seedable in which case they also have seed() member functions and corresponding constructors similar to those of Random Number Engines. Seeding a PRF is equivalent to setting its key to some value corresponding to the output of sseq.generate(). A PRF has a Countable Domain if its domain_type has member functions begin(), end(), and back() with semantics identical to that of boost::array<Uint, N> for some unsigned integral type Uint and some N. It must also have member functions domain_array_min() and domain_array_max() that specify the permissible domain values. Any array, da, whose members, da[i], satisfy: prf.domain_array_min() <= da[i] <= prf.domain_array_max[] must be "permissible". A PRF has an Integer Array Range if its range_type has member functions begin(), end(), and size() with semantics identical to boost::array<RangeInt, N>, for some integral type, RangeInt and constant N. It must also have static constexpr member functions, range_array_min() and range_array_max() that specify the subset of range values sampled by the PRF. For any sequence of permissible domain value, d1, d2, ..., the integers: prf(d1)[i], prf(d2)[i] must be statistically indistinguishable from independent uniformly distributed RangeInts from range_array_min() up to and including range_array_max(). If a PRF has a Countable Domain and an Integer Array Range, then: counter_based_urng<Prf> satisfies the requirements of a Uniform Random Number Generator. If, in addition, the PRF is Seedable, Streamable and Equality Comparable, then: counter_based_engine<Prf> satisfies the requirements of a Random Number Engine. A PRF is a Pseudo-Random Bijection if it is one-to-one and onto. I.e., if it has an inverse. It is not necessary for the inverse to have a practical implementation. It is sufficient that the inverse exist mathematically. A PRF is a Pseudo-Random Permutation if it is a Bijection and its domain_type is the same as its range_type. The four implemented PRFs are CopyAssignable, EqualityComparable, Streamable, Keyable, Seedable, Pseudo-Random Permutations with Integer Array ranges and Countable domains. Hence, they satisfy the requirements of counter_based_engine and counter_based_urng. Differences from Random123 -------------------------- This code is inspired by the Random123 library, but there are significant differences. New terminology: threefry<2, uint64_t> is a "Pseudo-Random Function" (PRF). PRFs take a key_type as a constructor argument, and are bijections on their ctr_type. Thus, instead of: typedef r123::Threefry2x64 r123_t; r123_t::ctr_type c; r123_t::key_type k r123_t::ctr_type r = r123_t(c, k) You now say: typedef boost::random::threefry<2, uint64_t> prf_t; prf_t::key_type k; prf_t myprf(k); prf_t::domain_type c; prf_t::range_type r = myprf(c); // Or alternatively, (more reminiscent of Random123) prf_t::range_type r2 = prf_t(k)(c); AESNI ----- Use of AESNI is tricky because we want both compile-time and runtime control over whether AESNI is used. The compile-time control is determined by BOOST_HAS_AESNI, which we assume will be set by some config file based on compiler-specific symbols (__AES__, __x86_64__, __ICC, __GNUC__) and possible user-overrides. But even if AESNI support is compiled in, we have to detect it and fall back to a software implementation at runtime if it's not available. The check requires either an asm with GCC-compatible compilers or an intrinsic with MSVC-compatible compilers. Endianness is another complication. The AESNI implementation only works on x86_64 platforms, so it is not endian-sensitive. In fact, it *defines* how multi-byte quantities are to be interpreted. In contrast, the software implementation has to be endian-aware. The current implementation has not been tested with other endians, and hence is almost certainly broken. And finally, the AESNI intrinsics are *very* fast. We don't want to weigh them down with lots of runtime checks. With AESNI enabled, the runtime overhead should cost no more than a single conditional at the top of operator()(c). The choice between hardware and software AES is controlled by a member data field, useAESNI, of the prf class. If hw AES is compiled in, useAESNI is initialized by a runtime check of the CPU capabilities. It can be changed with a mutator method. If hw AES is not compiled in, useAESNI is initialized false and it is an error to try to set it to true. In detail/aes_impl.hpp, we implement two classes: detail::hw128 detail::sw128 with a common interface that minimally meets the needs of AES and ARS. The interface includes +=, ^= constructors from array<uintW, N>, aesenc, and aeskeygenassist primitives, etc. The crucial functions in AES and ARS are then written with a template parameter that is assumed to provide the interface. An idiom like: #if BOOST_HAS_AESNI if( useAESNI ) method<detail::hw128>(args); else #endif method<detail::sw128>(args); gives the desired compile-time and runtime control.

7 10