December 2009 - Boost - lists.boost.org

[lockfree::fifo] Review
by Andrew Venikov 24 Jan '10

24 Jan '10

I decided to start a new thread on this topic since the last thread on the same topic is pretty outdated and not everyone will see it. (you can disregard my reply to the previous thread as this one is more complete) First of all I think that having a library of lock-free algorithms/containers would be greatly beneficial to a great number of boost users. On the other hand, having implemented several lock-free algorithms, I know that it is notoriously hard to get them right. We'll need all the help we can get from knowledgeable people in the field. The implementation needs to be thoroughly reviewed. Let me give the first attempt at that. In this e-mail I'll be primarily concentrating on the algorithm correctness rather than the interface. First, let me start with the choice of the algorithm for the fifo queues: Magued Michael and Michael Scott's "Simple, Fast, and Practical Non-blocking and blocking concurrent queues". I really think it is the best choice. Having said that, however, I need to point out that there some inherent limitation in this algorithm. The first limitation is that you can't peek at the head of the queue without dequeueing the element first, hence you can't get an interface that is conformant with std::queue. But I truly believe that it will be the case with any lock-free queue implementation, hence we should abandon the hope of having a lock-free queue look the same as std::queue Second, this algorithm makes certain assumptions about the ability to access memory that has been freed. The algorithm can try to read memory that has been de-allocated. (See further down for description of how). Several debug implementations of malloc/free store every allocated element on one page and when the element is freed, the page is marked protected for both reading and writing. That means that this algorithm can only work with a subset of custom allocators. Your "free_list" for example can't work with it since it's possible for a node to be freed when you call "deallocate". The "caching_freelist" and "static_freelist" on the other hand work just fine because they never return the memory back to the system. Third, and the biggest limitation, is the algorithm's assumption that you can store a pointer to an element plus an ABA tag in a contiguous memory space that can be passed to a CAS instruction. I believe not all platforms provide a DCAS instruction, that's why you're trying to work around this issue by using "packed pointers", i.e. storing the ABA tag in the part of the pointer that presumably is never used. The problem with that, however, is that some libc implementations of malloc/free do use all the bits in the pointer for their own needs, so you can't assume that you can meddle with them. My experience with this algorithm shows that the safest approach is to use a pre-allocated array for the queue and replace pointers with indexes into the array. Assuming that the size of an index is sufficiently lower than the size of a pointer, you can use the remaining bits to store the ABA tag. This alleviates both point 2 and 3. This does mean however, that the queue can't have an unbound size, the size has to be specified during the queue instantiation. Fourth, on platforms where CAS is implemented using LL/SC (such as PowerPC and MIPS), it is possible to livelock. The problem with LL/SC on most platforms is that the write granulum for marking LL's address as "dirty" isn't just the memory pointed to by LL, but the whole cache line where the memory resides. On these platforms it is important to make sure that each "node" of the queue is located on a separate cache line. Here's the code of your enqueue: bool enqueue(T const & t) { node * n = alloc_node(t); if (n == NULL) return false; for (;;) { atomic_node_ptr tail (tail_); //1 read_memory_barrier(); //2 atomic_node_ptr next (tail->next); //3 if (likely(tail == tail_)) //4 { if (next.get_ptr() == 0) //5 { if ( tail->next.cas(next, n) )//6 { tail_.cas(tail, n); return true; } } else tail_.cas(tail, next.get_ptr()); } } } 1. You're missing write barrier after a call to alloc_node(). Rational: alloc_node sets the "next" pointer of the newly allocated node to NULL. That is a store to shared memory. Setting next pointer to NULL has to be observable to other threads BEFORE they observe the inserted node. If you don't insert a write barrier, it is possible for other threads to see the new element in the list before element's next pointer is initialized with NULL. 2. You've misplaced the read_barrier. Rational: Line 4 checks the validity of the load done on line 3. That means that the read_barrier should be between lines 3 and 4. If you don't have a read_barrier after loading *tail->next, then this load can happen after the load of tail_ on line 4, which defies the purpose of line 4. Here's a scenario that makes it clear: State: queue is well-formed and tail_ points to the last element. 1. Thread A executes line1. State: the queue is unchanged and Thread A has a local copy of tail_ 2. Since there's no read barrier after line 3, loads at lines 3 and 4 happen out-of-order. tail_ is pre-loaded for comparison at line 4. 3. Thread B removes the element pointed by tail_, advances tail_ further State: tail_ is advanced, the old element is removed and given back to the allocator. A's local copy of tail_ becomes stale. 4. Thread A executes line 3 State: A's copy of tail is stale. This copy is used to load memory that has been reclamed by the allocator and possibly modified. The value of the memory happens to be 0. 5. Thread A executes line 4. Since tail_ has been preloaded, line 4 succedes. 6. Line 5 succedes 7. Line 6 succedes. Boom: you've just modified a piece of memory not controlled by the queue. You still need to have a barrier between lines 1 and 3, but here only a data dependency barrier will suffice, which is a weaker form of a read barrier and on most all platforms is a no-op since data dependent loads can be handled by processors automatically. 3. Your tail_ and head_ should be volatile. Furthermore, the getters for you tagged_ptr (like get_ptr, operator->() and operator* ) should return pointers/references to volatile types. Rational: If your tail_ is not volatile, then the compiler is free to load head_ just once and line 4 effectively becomes a no-op. Here's code for your "dequeue": bool dequeue (T * ret) { for (;;) { atomic_node_ptr head (head_); //1 read_memory_barrier(); //2 atomic_node_ptr tail(tail_); //3 node * next = head->next.get_ptr(); //4 if (likely(head == head_)) //5 { if (head.get_ptr() == tail.get_ptr()) //6 { if (next == 0) return false; tail_.cas(tail, next); } else { *ret = next->data; //7 if (head_.cas(head, next)) //8 { dealloc_node(head.get_ptr()); //9 return true; } } } } } 1. Again, you need to put read_barrier between lines 4 and 5. Only data dependency barrier will suffice at line 2. 2. You need to add write barrier after line 7 for the same reason as in the enqueue. Hope this was descriptive, Andy Venikov.

10 64

[typeof] Variadic functions
by Sergey Zubkovsky 22 Jan '10

22 Jan '10

Hi, Here is a patch to support variadic functions in the Boost.Typeof. Can anyone check it and commit? Thank you. Index: register_functions.hpp =================================================================== --- register_functions.hpp (revision 57837) +++ register_functions.hpp (working copy) @@ -18,18 +18,25 @@ #include BOOST_TYPEOF_INCREMENT_REGISTRATION_GROUP() #ifndef BOOST_TYPEOF_LIMIT_FUNCTION_ARITY -#define BOOST_TYPEOF_LIMIT_FUNCTION_ARITY 10 +#define BOOST_TYPEOF_LIMIT_FUNCTION_ARITY 15 #endif enum { FUN_ID = BOOST_TYPEOF_UNIQUE_ID(), - FUN_PTR_ID = FUN_ID + 1 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), - FUN_REF_ID = FUN_ID + 2 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), - MEM_FUN_ID = FUN_ID + 3 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), - CONST_MEM_FUN_ID = FUN_ID + 4 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), - VOLATILE_MEM_FUN_ID = FUN_ID + 5 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), - VOLATILE_CONST_MEM_FUN_ID = FUN_ID + 6 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY) + FUN_VAR_ID = FUN_ID + 1 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + FUN_PTR_ID = FUN_ID + 2 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + FUN_VAR_PTR_ID = FUN_ID + 3 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + FUN_REF_ID = FUN_ID + 4 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + FUN_VAR_REF_ID = FUN_ID + 5 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + MEM_FUN_ID = FUN_ID + 6 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + MEM_FUN_VAR_ID = FUN_ID + 7 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + CONST_MEM_FUN_ID = FUN_ID + 8 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + CONST_MEM_FUN_VAR_ID = FUN_ID + 9 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + VOLATILE_MEM_FUN_ID = FUN_ID + 10 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + VOLATILE_MEM_FUN_VAR_ID = FUN_ID + 11 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + VOLATILE_CONST_MEM_FUN_ID = FUN_ID + 12 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY), + VOLATILE_CONST_MEM_FUN_VAR_ID = FUN_ID + 13 * BOOST_PP_INC(BOOST_TYPEOF_LIMIT_FUNCTION_ARITY) }; BOOST_TYPEOF_BEGIN_ENCODE_NS Index: register_functions_iterate.hpp =================================================================== --- register_functions_iterate.hpp (revision 57837) +++ register_functions_iterate.hpp (working copy) @@ -6,6 +6,12 @@ #define n BOOST_PP_ITERATION() +#if ( n > 1 ) +# define comma , +#else +# define comma +#endif + // function pointers template<class V, class R BOOST_PP_ENUM_TRAILING_PARAMS(n, class P)> @@ -15,6 +21,13 @@ typedef BOOST_TYPEOF_ENCODE_PARAMS(BOOST_PP_INC(n), FUN_PTR_ID + n) type; }; +template<class V, class R BOOST_PP_ENUM_TRAILING_PARAMS(n, class P)> +struct encode_type_impl<V, R(*)(BOOST_PP_ENUM_PARAMS(n, P)comma ...)> +{ + typedef R BOOST_PP_CAT(P, n); + typedef BOOST_TYPEOF_ENCODE_PARAMS(BOOST_PP_INC(n), FUN_VAR_PTR_ID + n) type; +}; + template<class Iter> struct decode_type_impl<boost::mpl::size_t<FUN_PTR_ID + n>, Iter> { @@ -24,6 +37,15 @@ typedef BOOST_PP_CAT(iter, BOOST_PP_INC(n)) iter; }; +template<class Iter> +struct decode_type_impl<boost::mpl::size_t<FUN_VAR_PTR_ID + n>, Iter> +{ + typedef Iter iter0; + BOOST_TYPEOF_DECODE_PARAMS(BOOST_PP_INC(n)) + typedef BOOST_PP_CAT(p, n)(*type)(BOOST_PP_ENUM_PARAMS(n, p)comma ...); + typedef BOOST_PP_CAT(iter, BOOST_PP_INC(n)) iter; +}; + #ifndef BOOST_TYPEOF_NO_FUNCTION_TYPES // function references @@ -35,6 +57,13 @@ typedef BOOST_TYPEOF_ENCODE_PARAMS(BOOST_PP_INC(n), FUN_REF_ID + n) type; }; + template<class V, class R BOOST_PP_ENUM_TRAILING_PARAMS(n, class P)> + struct encode_type_impl<V, R(&)(BOOST_PP_ENUM_PARAMS(n, P)comma ...)> + { + typedef R BOOST_PP_CAT(P, n); + typedef BOOST_TYPEOF_ENCODE_PARAMS(BOOST_PP_INC(n), FUN_VAR_REF_ID + n) type; + }; + template<class Iter> struct decode_type_impl<boost::mpl::size_t<FUN_REF_ID + n>, Iter> { @@ -53,6 +82,13 @@ typedef BOOST_TYPEOF_ENCODE_PARAMS(BOOST_PP_INC(n), FUN_ID + n) type; }; + template<class V, class R BOOST_PP_ENUM_TRAILING_PARAMS(n, class P)> + struct encode_type_impl<V, R(BOOST_PP_ENUM_PARAMS(n, P)comma ...)> + { + typedef R BOOST_PP_CAT(P, n); + typedef BOOST_TYPEOF_ENCODE_PARAMS(BOOST_PP_INC(n), FUN_VAR_ID + n) type; + }; + template<class Iter> struct decode_type_impl<boost::mpl::size_t<FUN_ID + n>, Iter> { @@ -62,6 +98,15 @@ typedef BOOST_PP_CAT(iter, BOOST_PP_INC(n)) iter; }; + template<class Iter> + struct decode_type_impl<boost::mpl::size_t<FUN_VAR_ID + n>, Iter> + { + typedef Iter iter0; + BOOST_TYPEOF_DECODE_PARAMS(BOOST_PP_INC(n)) + typedef BOOST_PP_CAT(p, n)(type)(BOOST_PP_ENUM_PARAMS(n, p)comma ...); + typedef BOOST_PP_CAT(iter, BOOST_PP_INC(n)) iter; + }; + #endif//BOOST_TYPEOF_NO_FUNCTION_TYPES #ifndef BOOST_TYPEOF_NO_MEMBER_FUNCTION_TYPES @@ -83,5 +128,6 @@ #define BOOST_TYPEOF_id VOLATILE_CONST_MEM_FUN_ID #include <boost/typeof/register_mem_functions.hpp> +#undef comma #undef n #endif

2 3

[BoostCon 2010] Final Call for Participation
by Hartmut Kaiser 20 Jan '10

20 Jan '10

Please distribute! -------------------------------- 4th Annual Boost Conference 2010 -------------------------------- Aspen CO, USA, May 10-14, 2010, www.boostcon.com Final Call for Participation -------------------------------- Important dates: Proposal submissions Extended to January 3, 2010 Proposals decisions sent January 25, 2010 (tentative program available) Fully scheduled program available February 10, 2010 Session materials due April 15, 2010 -------------------------------- We invite you to submit session proposals to the 4th Annual Boost Conference: BoostCon 2010 (Aspen CO, USA, May 10-14, 2010). Traditionally the main face-to-face event for all things Boost (www.boost.org) BoostCon 2010 will present leading speakers from the whole C++ community. From using the Boost libraries to writing and maintaining them, from evangelizing to deploying Boost within your organization, from infrastructure and process to vision and mission, and the new C++ Standard, BoostCon brings together the sessions, the colleagues, and the inspiration to support your work with C++ and Boost in particular for the next year. To reflect the breadth of the Boost community, the conference includes sessions aimed at two constituencies: Boost end-users and hard-core Boost library and tool developers. The program fosters interaction and engagement within and across those two groups, with an emphasis on hands-on, participatory sessions. Session topics -------------- Topics of interest include, but are not restricted to, the following: * General tutorial sessions introducing one or more Boost libraries * In-depth sessions on using specific Boost libraries * Case studies on using Boost * Experts panels * Advanced sessions on implementation techniques used within Boost libraries * C++0x and how it will change life for users and library writers * Development workshops to extend or enhance existing Boost libraries * Workshops on design process * Infrastructure workshops such as Build tools, Website, Testing * Concepts and Generic Programming * Hardware and infrastructure presentations focused on how libraries can make better use of the technology * Other topics likely to be of great interest to Boost users and developers Interactive and collaborative sessions are encouraged, as this is the nature of both the on-line Boost community and the style of learning and participation that has proven most successful at such events. Sessions can be tutorial based, with an emphasis on interaction and participant involvement, or workshop based, whether hands-on programming or paper-based, discussion-driven collaborative work. Session formats --------------- Presentations Presentations focus on a practitioner's ideas and experience with anything relevant to Boost and Boost users. Panels Panels feature three or four people presenting their ideas and experiences relating to Boost's relevant, controversial, emerging, or unresolved issues. Panels may be conducted in several ways, such as comparative, analytic, or historic. Tutorials Tutorials are sessions at which instructors teach conference participants specific Boost-relevant skills. Workshops Workshops provide an active arena for advancements in Boost- relevant topics. Workshops provide the opportunity for experienced practitioners to develop new ideas about a topic of common interest and experience. Author's Corner Presentations These were introduced at BoostCon 2008, and were a great success. They are short (30 minute) sessions, focusing on tips on usage and design. In addition, we're looking to uncover the hidden design gems in Boost libraries. Tool Vendors Presentations We actively encourage tool vendors and ISP's to submit proposals for a special Tool Vendors Session Track aimed at products related to Boost and C++ (compilers, libraries, tools, etc.). Other formats may also be of interest. Don't hold back a proposal just because it doesn't fit into a pigeonhole. Submitting a proposal --------------------- Standard Sessions are 90 minutes. You may submit a proposal for fractions or multiples of 90-minutes. Fractional proposals will be grouped into 90 minute sessions covering related topics. Longer sessions, such as tutorials and classes, will be assigned 90 minute, three hour (i.e. half day), or six hour (i.e. full day) time slots. Please include: * The working title. * Type of session: presentation, panel, tutorial, workshop, authors corner, vendor track, other. * A paragraph or two describing the topic covered, suitable for the conference web site. * Proposed length: 10-20 minute short talks, 45 minutes, 90 minutes, half day, full day. * Alternate lengths, if you are willing to make adjustments: 10-20 minute short-talks, 45 minutes, 90 minutes, half-day, full day. * Audience: users, developers, both. * Level: basic, intermediate, advanced. * A biography, suitable for the conference web site. * Your contact information (will not be made public). Submission details ------------------ All submissions have to be done through the EasyChair conference management system: http://www.easychair.org/conferences/?conf=boostcon10. If you have not already registered at EasyChair, you will need to do so in order to submit your proposal. All submissions will go through a peer review process. Authors are invited (but are not required) to submit PDF versions of full papers of up to 10 pages in ACM conference proceedings format (see http://www.acm.org/sigs/publications/proceedings-templates) The full papers are not required unless you want them published in the proceedings. All accepted proposals will be made available in the Association for Computing Machinery (ACM) Digital Library (approval pending). Best papers, after further reviews, will be considered to be book chapters or journal articles in a renowned journal. The session materials go on the BoostCon website and will be available to attendees. For general information on the BoostCon 2010 paper submission or the scope of technical papers solicited, please refer to the conference website at www.boostcon.com. For any other questions about the submission process or paper format, please contact the Program Committee at boostcon10(a)easychair.com. If you have any technical problems with EasyChair, please contact EasyChair for help. Note: Presenters must agree to grant a non-exclusive perpetual license to publish submitted materials, either electronically or in print, in any media related to BoostCon. Hartmut Kaiser, email: hartmut.kaiser(a)gmail.com (Program Committee Chair) David Abrahams, email: dave(a)boostpro.com (Conference Chair) On behalf of the conference organizers

2 1

The boost component dependencies blues
by Stefan Seefeld 19 Jan '10

19 Jan '10

I'm not sure what the best place is for this question, but since it is symptomatic for a general theme (which I have complained about a lot in the past), I'll write it here: I'm trying to generate a boost subset that allows me to build boost.python and its dependencies into my own package. To do that, I ran "bcp python ...". I'm surprised to find all these components in the generated source tree: date_time, graph, mpi, regex, serialization, test, thread. And that's only compiled components (content of lib/). There are many more header-only components (i.e. subdirectories under boost/), such as spirit, variant, property_map, property_tree, dynamic_bitset. Is this really a minimal subset to use boost.python ? I can't quite believe it. In particular, what is boost.mpi doing there, and what boost.serialization (and boost.spirit) ? I'm going to great length to be able to use boost in a production environment, but I have to admit that this is quite unacceptable. :-( Stefan -- ...ich hab' noch einen Koffer in Berlin...

9 24

[math] Comeau can't match math::tools::fmod_workaround
by Mateusz Loskot 19 Jan '10

19 Jan '10

Hi, I'm not sure if I should submit this issue directly to Trac, so I decided to post it here first. Compiling Boost.Math using Comeau + GCC on Linux throws error that suggests mismatch of dobule and long double where single type T is expected: como-linux.compile.c++ ../../../bin.v2/libs/math/build/como-linux/debug/link-static/ellint_3l.o Comeau C/C++ 4.3.10.1 (May 7 2008 12:23:21) for LINUX_INTEL_ELF_Beta Copyright 1988-2008 Comeau Computing. All rights reserved. MODE:non-strict warnings C++ noC++0x_extensions "../../../boost/math/special_functions/ellint_3.hpp", line 185: error: no instance of function template "boost::math::tools::fmod_workaround" matches the argument list argument types are: (double, long double) T rphi = boost::math::tools::fmod_workaround(fabs(phi), T(constants::pi<T>() / 2)); ^ detected during instantiation of "T boost::math::detail::ellint_pi_imp(T, T, T, T, const Policy &) [with T=long double, Policy=c_policies::c_policy]" at line 309 1 error detected in the compilation of "../../../libs/math/build/../src/tr1/ellint_3l.cpp". como -tused -c --long_long -DBOOST_ALL_NO_LIB=1 -D_GNU_SOURCE=1 --no_inlining -O0 -g --exceptions -I"../../.." -I"../../../libs/math/src/tr1" -o "../../../bin.v2/libs/math/build/como-linux/debug/link-static/ellint_3l.o" "../../../libs/math/build/../src/tr1/ellint_3l.cpp" Could any one confirm if it is an error indeed? Should I submit ticket? Best regards, -- Mateusz Loskot, http://mateusz.loskot.net Charter Member of OSGeo, http://osgeo.org

2 10

Review Wizard report for December 2009
by John Phillips 14 Jan '10

14 Jan '10

All, Please check the report below and let Ron and I know if there are any issues so we can correct them before putting it on the web site. Pay especial attention to the need for many qualified and interested Review Managers. John ============================================== Review Wizard Status Report for December 2009 ============================================== News ==== Polygon Library Accepted Boost 1.40 Released New Libraries: None Revised Libraries: Accumulators, Asio, Circular Buffer, Filesystem, Foreach, Function, Fusion, Hash, Interprocess, Intrusive, MPL, Program Options, Proto, Python, Serialization, Unordered, Xpressive Geometry Library Accepted Boost 1.41 Released New Libraries: Property Tree Revised Libraries: DateTime, Filesystem, Iostreams, Math, Multi-index Containers, Proto, Python, Regex, Spirit, System, Thread, Unordered, Utility, Wave, Xpressive MSM Library Review Underway Constrained Value Review - Review Result still Pending Older Issues ============ The Time Series Library, accepted in August 2007, has not yet been submitted to SVN. The Floating Point Utilities Library, has not yet been submitted to SVN. It is slated to be integrated with the Boost.Math library. The Switch Library, accepted provisionally in January 2008, has not yet been submitted for mini-review and full acceptance. The Phoenix Library, accepted provisionally in September 2008, has not yet been submitted for mini-review and full acceptance. For libraries that are still waiting to get into SVN, please get them ready and into the repository. The developers did some great work making the libraries, so don't miss the chance to share that work with others. General Announcements ===================== As always, we need experienced review managers. The review queue has been growing substantially but we have had few volunteers, so manage reviews if possible and if not please make sure to watch the review schedule and participate. Please take a look at the list of libraries in need of managers and check out their descriptions. In general review managers are active boost participants or library contributors. If you can serve as review manager for any of them, email Ron Garcia or John Phillips, "garcia at osl dot iu dot edu" and "phillips at mps dot ohio-state dot edu" respectively. We are also suffering from a lack of reviewers. While we all understand time pressures and the need to complete paying work, the strength of Boost is based on the detailed and informed reviews submitted by you. A recent effort is trying to secure at least five people who promise to submit reviews as a precondition to starting the review period. Consider volunteering for this and even taking the time to create the review as early as possible. No rule says you can only work on a review during the review period. A link to this report will be posted to www.boost.org. If you would like us to make any modifications or additions to this report before we do that, please email Ron or John. If you're a library author and plan on submitting a library for review in the next 3-6 months, send Ron or John a short description of your library and we'll add it to the Libraries Under Construction below. We know that there are many libraries that are near completion, but we have hard time keeping track all of them. Please keep us informed about your progress. The included review queue isn't a classic queue. It is more an unordered list of the libraries awaiting review. As such, any library in the queue can be reviewed once the developer is ready and a review manager works with the wizards and the developer to schedule a review. It is not FIFO. Review Queue ============ * Lexer * Shifted Pointer * Logging * Log * Join * Pimpl * Task * Endian * Conversion * Sorting * GIL.IO * AutoBuffer * String Convert * Move * Containers * Interval Containers * Type Traits Extensions * Interthreads * Bitfield * Lockfree -------------------- Lexer ----- :Author: Ben Hanson :Review Manager: Eric Neibler :Download: `Boost Vault <http://boost-consulting.com/vault/index.php?action=downloadfile&filename=bo…>`__ :Description: A programmable lexical analyser generator inspired by 'flex'. Like flex, it is programmed by the use of regular expressions and outputs a state machine as a number of DFAs utilising equivalence classes for compression. Shifted Pointer --------------- :Author: Phil Bouchard :Review Manager: Needed :Download: `Boost Vault <http://www.boost-consulting.com/vault/index.php?&direction=0&order=&directo…>`__ :Description: Smart pointers are in general optimized for a specific resource (memory usage, CPU cycles, user friendliness, ...) depending on what the user need to make the most of. The purpose of this smart pointer is mainly to allocate the reference counter (or owner) and the object itself at the same time so that dynamic memory management is simplified thus accelerated and cheaper on the memory map. Logging ------- :Author: John Torjo :Review Manager: Gennadiy Rozental :Download: http://torjo.com/log2/ :Description: Used properly, logging is a very powerful tool. Besides aiding debugging/testing, it can also show you how your application is used. The Boost Logging Library allows just for that, supporting a lot of scenarios, ranging from very simple (dumping all to one destination), to very complex (multiple logs, some enabled/some not, levels, etc). It features a very simple and flexible interface, efficient filtering of messages, thread-safety, formatters and destinations, easy manipulation of logs, finding the best logger/filter classes based on your application's needs, you can define your own macros and much more! Log --- :Author: Andrey Semashev :Review Manager: Needed :Download: `Boost Vault <http://tinyurl.com/cm9lum>`__ :Description: The library is aimed to help adding logging features to applications. It provides out-of-box support for many widely used capabilities, such as formatting and filtering based on attributes, sending logs to a syslog server or to Windows Event Log, or simply storing logs into files. It also provides basic support for the library initialization from a settings file. The library can also be used for a wider range of tasks and implement gathering and processing statistical information or notifying user about application events. Join ---- :Author: Yigong Liu :Review Manager: Needed :Download: http://channel.sourceforge.net/ :Description: Join is an asynchronous, message based C++ concurrency library based on join calculus. It is applicable both to multi-threaded applications and to the orchestration of asynchronous, event-based applications. It follows Comega's design and implementation and builds with Boost facilities. It provides a high level concurrency API with asynchronous methods, synchronous methods, and chords which are "join-patterns" defining the synchronization, asynchrony, and concurrency. Pimpl ----- :Author: Vladimir Batov :Review Manager: Needed :Download: | `Boost Vault <http://www.boost-consulting.com/vault/index.php?action=downloadfile&filenam…>`__ | http://www.ddj.com/cpp/205918714 (documentation) :Description: The Pimpl idiom is a simple yet robust technique to minimize coupling via the separation of interface and implementation and then implementation hiding. This library provides a convenient yet flexible and generic deployment technique for the Pimpl idiom. It's seemingly complete and broadly applicable, yet minimal, simple and pleasant to use. Task ----------- :Author: Oliver Kowalke :Review Manager: Needed :Download: `Boost Vault <http://www.boostpro.com/vault/index.php?action=downloadfile&filename=bo…>`__ :Description: Formerly called Thread Pool The library provides: - thread creation policies: determines the management of worker threads: - fixed set of threads in pool - create workerthreads on demand (depending on context) - let worker threads ime out after certain idle time - channel policies: manages access to queued tasks: - bounded channel with high and low watermark for queuing tasks - unbounded channel with unlimited number of queued tasks - rendezvous syncron hand-over between producer and consumer threads - queueing policy: determines how tasks will be removed from channel: - FIFO - LIFO - priority queue (attribute assigned to task) - smart insertions and extractions (for instance remove oldest task with certain attribute by newst one) - tasks can be chained and lazy submit of taks is also supported (thanks to Braddocks future library). - returns a task object from the submit function. The task it self can be interrupted if its is cooperative (means it has some interruption points in its code -> ``this_thread::interruption_point()`` ). Endian ------ :Author: Beman Dawes :Review Manager: Needed :Download: http://mysite.verizon.net/beman/endian-0.10.zip :Description: Conversion ---------- :Author: Vicente Botet :Review Manager: Needed :Download: `Boost Vault <http://www.boostpro.com/vault/index.php?action=downloadfile&filename=co…>`__ :Description: Generic explicit conversion between unrelated types. Boost.Conversion provides: * a generic ``convert_to`` function which can be specialized by the user to make explicit conversion between unrelated types. * a generic ``assign_to`` function which can be specialized by the user to make explicit assignation between unrelated types. * conversion between ``std::complex`` of explicitly convertible types. * conversion between ``std::pair`` of explicitly convertible types. * conversion between ``boost::optional`` of explicitly convertible types. * conversion between ``boost::rational`` of explicitly convertible types. * conversion between ``boost::interval`` of explicitly convertible types. * conversion between ``boost::chrono::time_point`` and ``boost::ptime``. * conversion between ``boost::chrono::duration`` and ``boost::time_duration``. Sorting ------- :Author: Steven Ross :Review Manager: Needed :Download: `Boost Vault <http://www.boostpro.com/vault/index.php?action=downloadfile&filename=al…>`__ :Description: A grouping of 3 templated hybrid radix/comparison-based sorting algorithms that provide superior worst-case and average-case performance to std::sort: integer_sort, which sorts fixed-size data types that support a rightshift (default of >>) and a comparison (default of <) operator. float_sort, which sorts standard floating-point numbers by safely casting them to integers. string_sort, which sorts variable-length data types, and is optimized for 8-bit character strings. All 3 algorithms have O(n(k/s + s)) runtime where k is the number of bits in the data type and s is a constant, and limited memory overhead (in the kB for realistic inputs). In testing, integer_sort varies from 35% faster to 8X as fast as std::sort, depending on processor, compiler optimizations, and data distribution. float_sort is roughly 7X as fast as std::sort on x86 processors. string_sort is roughly 2X as fast as std::sort. GIL.IO ------ :Author: Christian Henning :Review Manager: Needed :Download: `GIL Google Code Vault <http://gil-contributions.googlecode.com/files/rc2.zip>`__ :Description: I/O extension for boost::gil which allows reading and writing of/in various image formats ( tiff, jpeg, png, etc ). This review will also include the Toolbox extension which adds some common functionality to gil, such as new color spaces, algorithms, etc. AutoBuffer ---------- :Author: Thorsten Ottosen :Review Manager: Needed :Download: `Here <http://www.cs.aau.dk/~nesotto/boost/auto_buffer.zip>`__ :Description: Boost.AutoBuffer provides a container for efficient dynamic, local buffers. Furthermore, the container may be used as an alternative to std::vector, offering greater flexibility and sometimes better performance. String Convert -------------- :Author: Vladimir Batov :Review Manager: Needed :Download: `Boost Vault <http://www.boostpro.com/vault/index.php?action=downloadfile&filename=bo…>`__ :Description: The library takes the approach of boost::lexical_cast in the area of string-to-type and type-to-string conversions, builds on the past boost::lexical_cast experience and advances that conversion functionality further to additionally provide: * throwing and non-throwing conversion-failure behavior; * support for the default value to be returned when conversion fails; * two types of the conversion-failure check -- basic and better/safe; * formatting support based on the standard I/O Streams and the standard (or user-defined) I/O Stream-based manipulators (like std::hex, std::scientific, etc.); * locale support; * support for boost::range-compliant char and wchar_t-based string containers; * no DefaultConstructibility requirement for the Target type; * consistent framework to uniformly incorporate any type-to-type conversions. It is an essential tool with applications making extensive use of configuration files or having to process/prepare considerable amounts of data in, say, XML, etc. Move ---------------- :Author: Ion Gaztanaga :Review Manager: Needed :Download: http://svn.boost.org/svn/boost/sandbox/move/ and online documentation at http://svn.boost.org/svn/boost/sandbox/move/libs/move/doc/html/index.html :Description: In C++0x, move semantics are implemented with the introduction of rvalue references. They allow us to implement move() without verbosity or runtime overhead. Boost.Move is a library that offers tools to implement those move semantics not only in compilers with rvalue references but also in compilers conforming to C++03. Containers ------------------ :Author: Ion Gaztanaga :Review Manager: Needed :Download: http://www.boostpro.com/vault/index.php?action=downloadfile&filename=boost.… and online documentation at http://svn.boost.org/svn/boost/sandbox/move/libs/container/doc/html/index.h… :Description: Boost.Container library implements several well-known containers, including STL containers. The aim of the library is to offers advanced features not present in standard containers or to offer the latest standard draft features for compilers that comply with C++03. Interval Containers Library ------------------------- :Author: Joachim Faulhaber :Review Manager: Needed :Description: The Interval Template Library (Itl) provides intervals and two kinds of interval containers: Interval_sets and interval_maps. Interval_sets and maps can be used just as sets or maps of elements. Yet they are much more space and time efficient when the elements occur in contiguous chunks: intervals. This is obviously the case in many problem domains, particularly in fields that deal with problems related to date and time. Interval containers allow for intersection with interval_sets to work with segmentation. For instance you might want to intersect an interval container with a grid of months and then iterate over those months. Finally interval_maps provide aggregation on associated values, if added intervals overlap with intervals that are stored in the interval_map. This feature is called aggregate on overlap. It is shown by example: :: typedef set<string> guests; interval_map<time, guests> party; guests mary; mary.insert("Mary"); guests harry; harry.insert("Harry"); party += make_pair(interval<time>::rightopen(20:00, 22:00),mary); party += make_pair(interval<time>::rightopen_(21:00, 23:00),harry); // party now contains [20:00, 21:00)->{"Mary"} [21:00, 22:00)->{"Harry","Mary"} //guest sets aggregated on overlap [22:00, 23:00)->{"Harry"} As can be seen from the example an interval_map has both a decompositional behavior (on the time dimension) as well as a accumulative one (on the associated values). Type Traits Extensions -------------------------- :Author: Frédéric Bron :Review Manager: Needed :Download: https://svn.boost.org/trac/boost/browser/sandbox/type_traits :Description: The purpose of the addition is to add type traits to detect if types T and U are comparable in the sens of <, <=, >, >=, == or != operators, i.e. if t<u has a sens when t is of type T and u of type U (same for <=, >, >=, ==, !=). The following traits are added: is_equal_to_comparable<T,U> is_greater_comparable<T,U> is_greater_equal_comparable<T,U> is_less_comparable<T,U> is_less_equal_comparable<T,U> is_not_equal_to_comparable<T,U> The names are based on the corresponding names of the standard template library (<functional> header, section 20.3.3 of the standard). The code has the following properties: * returns true if t<u is meaningful and returns a value convertible to bool * returns false if t<u is meaningless. * fails with compile time error if t<u is meaningful and returns void (a possibility to avoid compile time error would be to return true with an operator, trick but this has little sens as returning false would be better) InterThreads ------------------- :Author: Vicente J. Botet Escriba :Review Manager: Needed :Download: `Boost Vault <http://www.boostpro.com/vault/index.php?action=downloadfile&filename=in…>`__ :Description: Boost.InterThreads extends Boost.Threads adding some features: * thread decorator: thread_decorator allows to define setup/cleanup functions which will be called only once by thread: setup before the thread function and cleanup at thread exit. * thread specific shared pointer: this is an extension of the thread_specific_ptr providing access to this thread specific context from other threads. As it is shared the stored pointer is a shared_ptr instead of a raw one. * thread keep alive mechanism: this mechanism allows to detect threads that do not prove that they are alive by calling to the keep_alive_point regularly. When a thread is declared dead a user provided function is called, which by default will abort the program. * thread tuple: defines a thread groupe where the number of threads is know statically and the threads are created at construction time. * set_once: a synchonizer that allows to set a variable only once, notifying to the variable value to whatever is waiting for that. * thread_tuple_once: an extension of the boost::thread_tuple which allows to join the thread finishing the first, using for that the set_once synchronizer. * thread_group_once: an extension of the boost::thread_group which allows to join the thread finishing the first, using for that the set_once synchronizer. (thread_decorator and thread_specific_shared_ptr) are based on the original implementation of threadalert written by Roland Schwarz. Boost.InterThreads extends Boost.Threads adding thread setup/cleanup decorator, thread specific shared pointer, thread keep alive mechanism and thread tuples. Bitfield --------------- :Author: Vicente J. Botet Escriba :Review Manager: Needed :Download: https://svn.boost.org/svn/boost/sandbox/bitfield with documentation available at https://svn.boost.org/svn/boost/sandbox/bitfield/libs/bitfield/doc/index.ht… :Description: Portable bitfields traits. Boost.Bitfield consists of: * a generic bitfield traits class providing generic getter and setter methods. * a BOOST_BITFIELD_DCL macro making easier the definition of the bitfield traits and the bitfield getter and setter functions. Lockfree ------------------ :Author: Tim Blechmann :Review Manager: Needed :Description: boost.lockfree provides implementations of lock-free data structures. lock-free data structures can be accessed by multiple threads without the necessity of blocking synchronization primitives such as guards. lock-free data structures can be used in real-time systems, where blocking algorithms may lead to high worst-case execution times, to avoid priority inversion, or to increase the scalability for multi-processor machines. boost.lockfree provides: * boost::lockfree::fifo, a lock-free fifo queue * boost::lockfree::stack, a lock-free stack the code is available from the boost vault: http://www.boostpro.com/vault/index.php?action=downloadfile&filename=boost_… and from my personal git repository: git://tim.klingt.org/boost_lockfree.git http://tim.klingt.org/git?p=boost_lockfree.git the documentation is available at: http://tim.klingt.org/boost_lockfree/ Libraries under development =========================== Please let us know of any libraries you are currently developing that you intend to submit for review. See https://svn.boost.org/trac/boost/wiki/LibrariesUnderConstruction for a current listing of libraries under development.

10 17

[msm] scalability
by Christophe Henry 12 Jan '10

12 Jan '10

Hi Simon, > I think it would be worthwhile talking a little more about compile time performance in the documentation. You're right, I could explain more in the documentation. Unfortunately, the answer is not straightforward. The compilation time (and how far the compiler plays with us) depends on a number of factors: 1) the compiler 2) the compiler version 3) even your hardware (I get very different compile times on 2 different machines with the same processor) 4) the state machine structure and the chosen front-end 1) usually, g++ crashes later than VC and needed fewer workarounds 2) Surprisingly, g++4.2 > g++ 4.4. VC10 > VC9 > VC8 3) you will need RAM, especially with VC8 4) the compile-time is proportional to the number of transitions in the transition table and to the depth of the state machine (meaning submachine of submachine of...). EUML will cost you much more with VC than g++ as it implies a metaprogramming layer above the standard metaprogramming of the standard front-end and because the underlying decltype/typeof seems much better implemented with g++ (if you don't count the generated code size). A small test with VC9 in Release mode (and no other code) gave me: 20 transitions => 11s 30 transitions => 16s 40 transitions => 22s 50 transitions => 31s This is just to give you an idea as it can greatly depend on the factors named above. > Also I'd like to see some suggestions of how to improve compile times (and how factors including the number of entries in a transition table affect the compile time). There is just one solution: reduce the number of transitions. The best way to do it is using orthogonal regions. For example, if you see in your diagram many transitions to a single state based on the same event, you should factor them all into a second region. Adding submachines could help in some cases. On one hand, the compiler now has to generate code for a second machine, OTOH it reduces the number of transitions in the main one. This will only help if your submachine handles a small number of events, as Msm will add a row in the main table for every event of the submachines. If you can in exchange reduce the number of transitions, it can help. > Is there provision to split a machine across multiple compilation units (I assume not)? You unfortunately assume correctly, as what costs compile-time really is the heavy metaprogramming. > You include instructions for how to increase the maximum size of an MPL vector, but this only works up to 50 entries. What do we need to do to increase this further? This is pretty simple, we need to: - add a vector60/70/80....hpp in mpl/vector. - add a map60/70....hpp in mpl/map. - if more than 50 states, add vector60/70....hpp in fusion/vector This is pretty simple. If that helps, I can add them into the Msm sandbox as I already wrote them. We could also request a feature in the MPL. You will however not manage to increase the size of the transition table forever. At some point will almost every compiler crash in a big cloud of dark smoke. To give you an idea, I usually manage: - 60-80 transitions max with VC9. The exact point is variable on a number of different factors which are hard to understand. - probably less with VC8. VC8 will also quite fast need > 2GB RAM. - 80 transitions with g++4.2. It could probably manage more but I ran out of RAM. - 50-60 with g++4.4. Why 4.4 is in this case less good than 4.2 is a mystery to me. - In Debug mode, you'll see the compiler give up faster - Euml will also have an influence. > Euml looks really cool, but I'm getting an ICE when using it with 36 transitions in EumlSimple.cpp (12 and 24 work fine though) VC8 only has limited support for eUML for 3 reasons: - less than perfect metaprogramming capabilities - bad support for SFINAE - VC problems with decltype/typeof So yes, I would advise eUML for VC8 for only small state machines and without using complicated functors as actions/guards. VC9 would work much better, VC10 better than VC9 too (so Microsoft seems to be working on it). >One of my favourite features of boost::statechart is state local storage. It looks like this would have to be done manually in msm? Yes, this is not supported yet. However, Msm has always been developed very user-oriented and what users ask, they usually get it. Msm has from the beginning greatly gained from clever people suggesting new features and I will not stop the development after the review, whatever the outcome is. If you'd like a feature, I'd be happy to discuss a solution with you and implement it in the next version, thus making Msm more useful. > The library is looking really powerful and expressive (not to mention fast!) for small state machines. Thanks. This is the reason for the compile-times ;-) Christophe

5 5

Re: [boost] Meta State Machine (MSM) review
by Christophe Henry 12 Jan '10

12 Jan '10

Hi Barend, Sorry that I didn't have time to address earlier the questions in your review. >I would like to see a connection with Boost.Graph (I mean, see it in an >example or so, not within the kernel), to see what I'm doing with all >those transitions; I asked this on the list and this is possible, good. It is a good point and I'll try to provide this, at least for simple state machines. >1) the struct "Row" is written with a capital, this is unusual within >Boost. Words with a capital are normally template parameters. I >understand why it is done (there was already a row without a capital in >Msm 1.1), but I think it could have been solved differently (for >example: "transition", or if that is too long "tr" which could stand for >transition row or table row). I don't like to see Row where I expect to >see a normal classname. Conventions are always hard to define. I usually try to follow the following convention: - rows are usually not capitalized (a_row, g_row...) - except the row working with functors, like Row - eUML functors are also capitalized and finish with an underscore. - eUML functions are not capitalized and finish with an underscore. This tries to match the phoenix syntax. This allows differentiation between functors/non functors. I found this system practical but would not mind changing it if required. >The capitals are also used in the documentation and samples (struct >Empty) but there I've less problems with it. Maybe use of Capitals for >states is a state machine convention Yes, it is a common habit in UML to capitalize state names (though I am not aware of any written rules). >I noticed different conventions in >the source files. Many template parameters are completely capitalized. I >had done the same thing, earlier, but was hinted that this is unusual as >well. Just a matter of style (as long as library users normally don't >see it, it is less important) Yes, while the user doesn't see it, it is not very beautiful so I'll change this. >2) the class state_machine is quite long (in lines of code), containing >several sub-structs which maybe (don't know) could be moved to a >separate source file (detail). Yes, it became quite big, you're right. It's quite practical for me to have functions and helpers at the same place but some parts could be separated. >You have to call p.start() manually Yes. Calling start() will move the machine in its initial state and call on_entry on the machine and the initial state. You have to call it explicitly because sometimes you simply don't want on_entry to be called at the creation of the state machine. >Thanks for submitting this excellent library. I'm happy that you like it. Thanks for the great work you put on the review. Regards, Christophe

2 1

Re: [boost] [msm] eUML guard/action location
by Christophe Henry 10 Jan '10

10 Jan '10

>Are all those parentheses used for optional arguments, or could we consider eliminating them? I'd love to eliminate them but don't see how. They actually are constructors. The complete expression would be something like: typedef BOOST_TYPEOF(build_stt(( DestState() = CurrentState() + cool_event()[guard()]/action(), DestState2() = CurrentState() + cool_event()[guard2()]/action2() ) ) ) transition_table; So I'm really just pretending to pass on-the-fly instances of arguments from types DestState, CurrentState, etc. to evaluate the result with typeof after proto is done.

5 4

[Boost.Build] Unexpected MSVC linker error
by Sergey Zubkovsky 08 Jan '10

08 Jan '10

Hi, I'm using bjam and VC++2008 Express Edition to build date_time library, for example. When I try to link it with my project I have the error: LINK : fatal error LNK1104: cannot open file 'boost_date_time-vc90-mt-gd-1_40.lib' The bjam's command line options are : --layout=tagged toolset=msvc The name of the genereated lib file is "boost_date_time-mt-gd.lib". The error doesn't occur if the "--layout" option is "versioned" or "system". It seems to me that the cause of this behavior is the following text in the file "boost\config\auto_link.hpp": #ifndef BOOST_AUTO_LINK_NOMANGLE # pragma comment(lib, BOOST_LIB_PREFIX BOOST_STRINGIZE(BOOST_LIB_NAME) "-" BOOST_LIB_TOOLSET BOOST_LIB_THREAD_OPT BOOST_LIB_RT_OPT "-" BOOST_LIB_VERSION ".lib") # ifdef BOOST_LIB_DIAGNOSTIC # pragma message ("Linking to lib file: " BOOST_LIB_PREFIX BOOST_STRINGIZE(BOOST_LIB_NAME) "-" BOOST_LIB_TOOLSET BOOST_LIB_THREAD_OPT BOOST_LIB_RT_OPT "-" BOOST_LIB_VERSION ".lib") # endif #else # pragma comment(lib, BOOST_STRINGIZE(BOOST_LIB_NAME) ".lib") # ifdef BOOST_LIB_DIAGNOSTIC # pragma message ("Linking to lib file: " BOOST_STRINGIZE(BOOST_LIB_NAME) ".lib") # endif #endif "-" BOOST_LIB_VERSION ".lib" is always added in the "pragma comment" directive, if BOOST_AUTO_LINK_NOMANGLE is undefined, of cause. Is this a bug? How can I workaround it if this is not a bug? Thank you.

4 6