September 2014 - Boost-users

[Thread] thousands of spurious wakeups from timed_wait() per second
by Steven Clark 15 Sep '14

15 Sep '14

I've been trying to get useful behavior out of condition_variable::timed_wait() with little success. Depending on the platform, I'm getting about 3000 or nearly 100000 spurious wakeups per second. I'm working on a few different Linux platforms with different versions of Boost: 1.46, 1.48, and another in the high 40's. The eventual target is 1.48. I've looked for bug reports and haven't found any, and I have no evidence yet that suggests the different versions are significant to this problem. Because of the lack of problem reports, I figure I'm doing something wrong. That's my question: "What am I doing wrong here?" I have a test program that sets up a typical producer/consumer example. The main program spawns a thread, the producer, that notifies the main thread via a condition_variable once a second for a few seconds, then ends. The main thread, the consumer, waits efficiently for notifications with a 2-second timeout. When the producer thread finishes, the main (consumer) thread eventually times out, joins the spawned thread, and exits. At least, that's what is supposed to happen. If I don't bother with timeouts and use condition_variable::wait(), it works perfectly - except for the timeout, join, and exit. This is as expected. So far I have seen no spurious wakeups at all with this test. There is an ominous yet ambiguous remark in the documentation "The duration overload of timed_wait is difficult to use correctly. The overload taking a predicate should be preferred in most cases." Due to that remark I stopped experimenting with relative timeouts before I got anything to work. Using absolute timeouts, I get very high numbers of spurious wakeups as mentioned above - 3000 and about 100000 per second depending on the platform. I see no difference between timed_wait with a predicate and without a predicate. I'm also having trouble with the return value of timed_wait(). The documentation carefully states that timed_wait() returns "false if the call is returning because the time specified by abs_time was reached, true otherwise". This seems to mean that spurious wakeups return true -- and if you think about it, the return value is useless if spurious wakeups could return false. You'd have to add your own check for timeout. I was quite surprised to find that every spurious wakeup of timed_wait() that I've ever seen returns false. That just doesn't make sense to me. Here is the version of my test program that uses timed_wait without a predicate, followed by its output. #include <boost/thread.hpp> #include <boost/date_time.hpp> #include <iostream> boost::condition_variable gCond; boost::mutex gMutex; bool gGotData; unsigned gValue; void threadfn() { for (unsigned i = 0; i < 7; i++) { std::cerr << "hello from threadfn(), #" << i << std::endl; { boost::lock_guard<boost::mutex> guard(gMutex); gGotData = true; gValue = i; } gCond.notify_one(); boost::this_thread::sleep(boost::posix_time::seconds(1)); } } int main() { std::cerr << "******* Using timed_wait()" << std::endl; gGotData = false; gValue = 999; boost::thread t(threadfn); while (true) { boost::system_time deadline( boost::posix_time::microsec_clock::local_time()); std::cerr << "now is " << deadline << std::endl; deadline += boost::posix_time::seconds(2); std::cerr << "deadline is " << deadline << std::endl; boost::unique_lock<boost::mutex> lock(gMutex); unsigned nWakeups = 0; while(!gGotData) { bool ret = gCond.timed_wait(lock, deadline); nWakeups++; if (ret) std::cerr << "timed_wait() returned true on wakeup #" << nWakeups << std::endl; } std::cerr << "main() exited inner loop with " << nWakeups << " wakeups; gGotData is " << (gGotData ? "true" : "false") << std::endl; if (gGotData) { std::cerr << "main(), notified, gValue is " << gValue << std::endl; gGotData = false; } } // not reached in this incarnation of the code t.join(); std::cerr << "main(), after joining" << std::endl; } ******* Using timed_wait() now is 2014-Sep-09 20:48:12.183781 deadline is 2014-Sep-09 20:48:14.183781 hello from threadfn(), #0 timed_wait() returned true on wakeup #1451 main() exited inner loop with 1451 wakeups; gGotData is true main(), notified, gValue is 0 now is 2014-Sep-09 20:48:12.215471 deadline is 2014-Sep-09 20:48:14.215471 hello from threadfn(), #1 timed_wait() returned true on wakeup #96864 main() exited inner loop with 96864 wakeups; gGotData is true main(), notified, gValue is 1 now is 2014-Sep-09 20:48:13.216511 deadline is 2014-Sep-09 20:48:15.216511 hello from threadfn(), #2 timed_wait() returned true on wakeup #95911 main() exited inner loop with 95911 wakeups; gGotData is true main(), notified, gValue is 2 now is 2014-Sep-09 20:48:14.216805 deadline is 2014-Sep-09 20:48:16.216805 hello from threadfn(), #3 timed_wait() returned true on wakeup #97526 main() exited inner loop with 97526 wakeups; gGotData is true main(), notified, gValue is 3 now is 2014-Sep-09 20:48:15.217191 deadline is 2014-Sep-09 20:48:17.217191 hello from threadfn(), #4 timed_wait() returned true on wakeup #104710 main() exited inner loop with 104710 wakeups; gGotData is true main(), notified, gValue is 4 now is 2014-Sep-09 20:48:16.217453 deadline is 2014-Sep-09 20:48:18.217453 hello from threadfn(), #5 timed_wait() returned true on wakeup #102404 main() exited inner loop with 102404 wakeups; gGotData is true main(), notified, gValue is 5 now is 2014-Sep-09 20:48:17.217685 deadline is 2014-Sep-09 20:48:19.217685 hello from threadfn(), #6 timed_wait() returned true on wakeup #97906 main() exited inner loop with 97906 wakeups; gGotData is true main(), notified, gValue is 6 now is 2014-Sep-09 20:48:18.217291 deadline is 2014-Sep-09 20:48:20.217291 ^C Thank you very much, Steven Clark

4 9

[fusion] BOOST_FUSION_DEFINE_STRUCT with user defined methods
by Tobias Loew 15 Sep '14

15 Sep '14

Hi, I often use BOOST_FUSION_DEFINE_STRUCT not to generate a fusion-sequence per se, but to get a struct with - default initialisation for all members - a reasonable set of constructors - equality-check - all above without additional code But sometimes I would like to have additional methods in those structs, but deriving from those or fusion-adapting them loses at least one of the nice properties above. After looking at the BOOST_FUSION_DEFINE_STRUCT implementation I though about splitting the makro BOOST_FUSION_DEFINE_STRUCT_IMPL an inject my code in-between. Now my questions are: - Is there anything I've overseen and might detroy some fusion-mechanism? - Is there any chance to get macros like BOOST_FUSION_DEFINE_STRUCT_[BEGIN|END] (and also for the other BOOST_FUSION_DEFINE_... macros? Tobias -- View this message in context: http://boost.2283326.n4.nabble.com/fusion-BOOST-FUSION-DEFINE-STRUCT-with-u… Sent from the Boost - Users mailing list archive at Nabble.com.

4 8

WG: [atomic] can it bechecked that boost::atomic<T> *v = reinterpret_cast<boost::atomic<T>*>(var); is safe?
by Tim Odenthal 15 Sep '14

15 Sep '14

On 15 Sep 2014, at 18:00, boost-users-request(a)lists.boost.org<mailto:boost-users-request@lists.boost.org> wrote: On Monday, September 15, 2014 07:31 PM, Tim Odenthal wrote: Dear experts, essentially, I use this type of reinterpret_cast for doubles, and it __seems__ to work fine. I compile-time check that the sizeof(boost::atomic<T>) == sizeof( T ), but is that enough to ensure that this will always work? Should/could I check alignment issues as well? Even if the "atomic_cast" function has a (small) run-time overhead, it would be very much worth it in my setting, where I actually want to safely add to a double at some location in memory. Obviously, I want to use boost::atomic to be system/compiler agnostic, since it's supposed to eventually run with clang, gcc, (linux, OS X) as well as MS compilers, and so far we didn't see the need to move to C++11... I'm not sure I can figure out what problem you're trying to solve. Well, essentially, I have an array of doubles that at exactly one point in the code needs to be added to where I cannot ensure that each thread only adds to its own part of the array, so I want to ensure that each addition there is executed atomically. I think the cast is probably "safe", but far less can be said of it when it's dereferenced. That would depend entirely on the type of var. Is *var actually a boost::atomic<T> in the first place? If it is, why are you erasing its type? No, as I was saying, it’s a double *. How can you be sure there is nobody else aliasing var or *var and then reading or writing it? That is actually a problem, but since I completely control that part of the code, I hope to not make that mistake ;-). If *var is not a boost::atomic<T>, reinterpret_cast isn't going to magically convert it, you have undefined behaviour. Of course, one effect of UB is that it __seems__ to work, until, presumably, it doesn’t. But what, then, is the difference between an atomic<double>* and a normal double*, if the sizes (and possibly alignments) are the same? Hmmm, I just read up on what a reinterpret_cast<> is supposed to be good for, which is very little indeed, so it seems that the behaviour is undefined - but then there is no way to solve that problem? (So far I only ever used it to print the address of a pointer...) Since openMP allows for a pragma atomic, somehow, it seems that there should be a way. Using a simple spin_lock is way too slow (tested), since it doesn’t lock a specific memory location, but essentially everything, and I suspect that more complicated locking schemes will be also more expensive, just for different reasons. Ben Thanks for your insight! Tim

2 1

[atomic] can it bechecked that boost::atomic<T> *v = reinterpret_cast<boost::atomic<T>*>(var); is safe?
by Tim Odenthal 15 Sep '14

15 Sep '14

Dear experts, essentially, I use this type of reinterpret_cast for doubles, and it __seems__ to work fine. I compile-time check that the sizeof(boost::atomic<T>) == sizeof( T ), but is that enough to ensure that this will always work? Should/could I check alignment issues as well? Even if the "atomic_cast" function has a (small) run-time overhead, it would be very much worth it in my setting, where I actually want to safely add to a double at some location in memory. Obviously, I want to use boost::atomic to be system/compiler agnostic, since it's supposed to eventually run with clang, gcc, (linux, OS X) as well as MS compilers, and so far we didn't see the need to move to C++11... Thanks for any advice in advance Tim

2 1

program_options Parsing Suboptions
by Robert 14 Sep '14

14 Sep '14

Hi, I am trying to use program_options to parse suboptions: Command –S “-a 1 –b test –c 1.2” (the suboptions in –S are to be passed into other programs) Using the example source code, I tried something like the code below, but get the following error The argument (‘ 1’) for option ‘—TestA’ is invalid. Apparently the space between –a and 1 is treated as part of the argument and thus will not convert to an integer. I tried at the top argument level (e.g. command –M 1) and it works as expected, so something about creating the vector of suboptions is confusing program_options. Any suggestions? Thanks very much. options_description desc("Allowed options"); desc.add_options()("subopt,S",value<string>(),"Sub Options"); variables_map vm; store(command_line_parser(ac, av).options(desc).run(), vm); vm.notify(); if (vm.count("subopt")) { string substring=vm["subopt"].as<string>(); boost::regex re("\\s+(?=-)"); boost::sregex_token_iterator i(substring.begin(), substring.end(), re, -1); boost::sregex_token_iterator j; vector<string> splitResults; copy(i,j,back_inserter(splitResults)); int ta; string tb; float tc; options_description subopt_desc("Suboptions"); subopt_desc.add_options() ("TestA,a", value<int>(&ta)->default_value(0), "integer") ("TestB,b",value<string>(&tb),"string") ("TestC,c",value<float>(&tc), "float") ; variables_map subopt_vm; store(command_line_parser(splitResults).options(subopt_desc).run(), subopt_vm); subopt_vm.notify(); cout<<"a="<<ta<<endl; cout<<"b="<<tb<<endl; cout<<"c="<<tc<<endl; } -- http://www.fastmail.fm - Choose from over 50 domains or use your own

1 1

[graph] how to find all vertices reachable from one edge of a starting vertex
by Michael Marcin 14 Sep '14

14 Sep '14

How can I find all vertices reachable by following one specified out-edge? Is there already an algorithm that does this? If not I think I could do it by: - marking all vertices white - mark starting vertex gray - depth_first_visit the target of the specified edge Does this work?

2 2

[container][slist] constant time splice_after
by Aaron Levy 14 Sep '14

14 Sep '14

What happens if I call slist::splice_after(this_iter, that_list, that_begin, that_end, nelems) with an nelems value that does not match std::distance(that_begin, that_end). In other words, is there any kind of validations performed on nelems and is it just used to update the size of the list? Aaron

2 1

how to construct an iterator_archetype<single_pass_traversal_tag>
by Peter Johansson 13 Sep '14

13 Sep '14

Hi, I'm writing tests using the iterator_archetype [http://www.boost.org/doc/libs/1_55_0/libs/iterator/doc/iterator_archetypes.…] and get confused how to use a single_pass_iterator such as e.g. iterator_archetype<int, readable_iterator_t, single_pass_iterator> The default constructor does not work as expected (since single_pass_iterators are not necessarily default constructible). The copy constructor is not very useful and that is all can find. I noticed the detail::ctor_arg and was hoping there were a constructor thaing that struct as argument - but no. How am I supposed to construct this class if I e.g. wanna passed it to a function that I intend to test. Thank you, Peter

1 0

[container][flat_map] Internal storage for flat_maps / flat_sets
by Aaron Levy 13 Sep '14

13 Sep '14

What is the type of the underlying contiguous storage used by the flat_map / flat_set containers? Does the following code make sense: flat_map<T> container; ... if (container.size() > 0) { const T& elem = *(container.begin()); const T& elem_1 = *(container.begin() + 1); assert(&elem_1 == &elem + 1); } Aaron

2 1

Problem linking with boost::program_options with clang 3.5 (OSX)
by Jürgen Simon 12 Sep '14

12 Sep '14

Hello, I require OpenMP on OSX and had to build OpenMP/Clang to that end. The compiler reads as clang 3.5. For some reason, the compiler seems to have a problem linking executables with boost. I tried boost version 1.55 but got a compilation error. Boost 1.56 compiles with this warning: clang-3.5: warning: argument unused during compilation: '-finline-functions' but other than that compiles through and installs just fine. When trying to link an executable with program_options however, I get the following linker error: Undefined symbols for architecture x86_64: "__ZN5boost15program_options11to_internalERKSs", referenced from: __ZN5boost15program_options11to_internalISsEESt6vectorISsSaISsEERKS2_IT_SaIS5_EE in radolan2cf.cpp.o "__ZN5boost15program_options19options_descriptionC1ERKSsjj", referenced from: _main in radolan2cf.cpp.o "__ZN5boost15program_options20invalid_option_valueC1ERKSs", referenced from: __ZN5boost15program_options8validateIfcEEvRNS_3anyERKSt6vectorISbIT0_St11char_traitsIS5_ESaIS5_EESaIS9_EEPT_l in radolan2cf.cpp.o "__ZN5boost15program_options22error_with_option_nameC2ERKSsS3_S3_i", referenced from: __ZN5boost15program_options16validation_errorC2ENS1_6kind_tERKSsS4_i in radolan2cf.cpp.o "__ZN5boost15program_options6detail7cmdline21set_additional_parserENS_9function1ISt4pairISsSsERKSsEE", referenced from: __ZN5boost15program_options25basic_command_line_parserIcE12extra_parserENS_9function1ISt4pairISsSsERKSsEE in radolan2cf.cpp.o "__ZN5boost15program_options6detail7cmdlineC2ERKSt6vectorISsSaISsEE", referenced from: __ZN5boost15program_options25basic_command_line_parserIcEC2EiPKPKc in radolan2cf.cpp.o "__ZN5boost15program_options8validateERNS_3anyERKSt6vectorISsSaISsEEPSsi", referenced from: __ZNK5boost15program_options11typed_valueISscE6xparseERNS_3anyERKSt6vectorISsSaISsEE in radolan2cf.cpp.o "__ZN5boost15program_optionslsERSoRKNS0_19options_descriptionE", referenced from: _main in radolan2cf.cpp.o "__ZNK5boost15program_options22abstract_variables_mapixERKSs", referenced from: __ZNK5boost15program_options13variables_mapixERKSs in radolan2cf.cpp.o "__ZNK5boost15program_options22error_with_option_name23substitute_placeholdersERKSs", referenced from: __ZTVN5boost16exception_detail10clone_implINS0_19error_info_injectorINS_15program_options20invalid_option_valueEEEEE in radolan2cf.cpp.o __ZTVN5boost16exception_detail19error_info_injectorINS_15program_options20invalid_option_valueEEE in radolan2cf.cpp.o __ZTVN5boost15program_options20invalid_option_valueE in radolan2cf.cpp.o __ZTVN5boost15program_options16validation_errorE in radolan2cf.cpp.o __ZTVN5boost16exception_detail10clone_implINS0_19error_info_injectorINS_15program_options16validation_errorEEEEE in radolan2cf.cpp.o __ZTVN5boost16exception_detail19error_info_injectorINS_15program_options16validation_errorEEE in radolan2cf.cpp.o "__ZNK5boost15program_options29value_semantic_codecvt_helperIcE5parseERNS_3anyERKSt6vectorISsSaISsEEb", referenced from: __ZTVN5boost15program_options11typed_valueIfcEE in radolan2cf.cpp.o __ZTVN5boost15program_options11typed_valueISscEE in radolan2cf.cpp.o ld: symbol(s) not found for architecture x86_64 clang-3.5: error: linker command failed with exit code 1 (use -v to see invocation) I’m a little stumped as to how to fix this problem. Can someone help me out? Kind Regards, Jürgen

1 0