- Boost - lists.boost.org

ACID transactions/flushing/MySQL InnoDB
by Stefan Strasser 12 Dec '09

12 Dec '09

hey, this is not a boost question, but it is for the development of a boost library, so maybe a database expert on the list could help me out? I've been benchmarking some test cases of my library against MySQL InnoDB, which isn't even apples to oranges anymore, but nevertheless gave some insights. what I find surprising is the performance of MySQL for small transactions. while my library spends 90% of its time waiting for the disk to flush a transaction, InnoDB uses 100% CPU and barely uses any disk. I've reduced the calls to fsync() for small transactions to 1 call per transaction after some sequential writing, so I'm wondering how MySQL accomplishes this? by definition there must be a flush per transaction, unless you loosen the requirements of the D in ACID. Is that what InnoDB is doing? the only way I see to reduce flushes even more is combining some transactions. but after a power failure some of those transactions can be lost even though they haven reported as successfully committed. I have not yet examined their source code yet. the MySQL documentation doesn't say anything about loosening Durability. (although you have to look pretty hard to see that the default isolation level isn't Serializable either.) thanks in advance, Stefan

5 17

[Boost-config] Add pgi-9.0 threading support
by Belcourt, Kenneth 12 Dec '09

12 Dec '09

Hi John, I've attached a patch to enable threading support with pgi-9.0 (I'm trying to resurrect the pgi tester). Does this look okay to commit? -- Noel

2 1

bug in boost::variant?
by Zachary Turner 11 Dec '09

11 Dec '09

I'm using boost 1.41. I use a variant to store a symbolic representation of an algebraic expression, wrapped inside of a master "node" class. Like this: struct expr_node { friend class constant; friend class variable; friend class arithmetic_op; friend class exponent; friend class special_func; friend class negative; public: template<class T> expr_node(const T& expr) : expr_(expr) { } expr_node& operator=(const expr_node& rhs) { expr_ = rhs; return *this; } private: typedef boost::variant<constant, variable, arithmetic_op, exponent, special_func, negative> node_type; node_type expr_; }; Of course, this is a recursive definition, because exponent, arithmetic_op, etc also contain expr_node members. At some point, I flatten the tree structure across either addition or multiplication, the result of which is a std::list<expr_node>. So in this example, if my expression is 2 * x * 2 * x^x * (2+x)^2 * (1 + x) this becomes the list [2, x, 2, x^x, (2+x)^2, 1+x]. I then use std::partition() to organize this list into the desired order according to the node type. This is where the strange behavior comes in. As you can see there are no negatives in this expression, and indeed, I can verify this is indeed the case in my code as well with: bool expr_node::is_negative() const { return boost::get<negative>(&expr_); } ... BOOST_ASSERT(std::find_if(terms.begin(), terms.end(), boost::bind(&expr_node::is_negative, _1)) == terms.end()); This assert passes. However, Immediately executing std::partition(terms.begin(), terms.end(), boost::bind(&expr_node::is_constant, _1)); BOOST_ASSERT(std::find_if(terms.begin(), terms.end(), boost::bind(&expr_node::is_negative, _1)) == terms.end()); triggers the assert. So something is going wrong involving copying the variants. Every class has both a copy constructor and an assignment operator. Examining the exact contents of the list both before and after the partition, the which_ fields are [0,1,0,3,3,2] (before), and [0,5,5,3,3,2] (after). Putting a breakpoint in all of the negative constructors, I can get a callstack for the first time a 'negative' is constructed. The callstack is as follows: negative::negative(const expr_node & arg={...}) Line 747 C++ node_type::convert_construct<expr_node const >(const expr_node & operand={...}, int __formal=1, int __formal=1) Line 1298 + 0xf bytes C++ node_type::node_type<expr_node>(const expr_node & operand={...}) Line 1367 C++ node_type::assign<expr_node>(const expr_node & rhs={...}) Line 1609 + 0xc bytes C++ node_type::operator=<expr_node>(const expr_node & rhs={...}) Line 1620 C++ expr_node::operator=(const expr_node & rhs={...}) Line 171 C++ std::swap<expr_node>(expr_node & _Left={...}, expr_node & _Right={...}) Line 23 C++ std::iter_swap<term_list::_Iterator<0>,term_list::_Iterator<0> >(term_list::_Iterator<0> _Left={...}, term_list::_Iterator<0> _Right={...}) Line 594 + 0x17 bytes C++ std::_Partition<term_list::_Iterator<0>,boost::_bi::bind_t<bool,boost::_mfi::cmf0<bool,expr_node>,boost::_bi::list1<boost::arg<1> > > >(term_list::_Iterator<0> _First={...}, term_list::_Iterator<0> _Last={...}, boost::_bi::bind_t<bool,boost::_mfi::cmf0<bool,expr_node>,boost::_bi::list1<boost::arg<1> > > _Pred={...}) Line 1838 + 0x4d bytes C++ std::partition<term_list::_Iterator<1>,boost::_bi::bind_t<bool,boost::_mfi::cmf0<bool,expr_node>,boost::_bi::list1<boost::arg<1> > > >(term_list::_Iterator<1> _First={expr_={...} }, term_list::_Iterator<1> _Last={expr_={...} }, boost::_bi::bind_t<bool,boost::_mfi::cmf0<bool,expr_node>,boost::_bi::list1<boost::arg<1> > > _Pred={...}) Line 1847 + 0x64 bytes C++ expr_node::organize() Line 213 + 0x77 bytes C++ Perhaps it's related to the implementation of swap() for variants? If a more detailed code sample is needed, I can upload this entire program to a web server for someone to download. Zach

2 2

Re: [boost] [MSM] compile errors
by Christophe Henry 11 Dec '09

11 Dec '09

Hi Michael, >FWIW. gcc-4.3.3 compiles SimpleTutorial.cpp just fine but has issues with >SimpleTutorialEuml.cpp. Pages and pages of a compiler babble about >boost::fusion::void_ and mpl_::na. > >Should I not be using boot 1.41? I just tried the version with old syntax and with the new one, with gcc-4.4 and boost 1.41 and got no warning, even in pedantic mode (excluding a few in boost/integer_traits.hpp), so I'm scratching my head. Christophe

1 0

interprocess named condition
by Jan 11 Dec '09

11 Dec '09

I'm guessing a named condition has to be used in a shared_memory_object, with a mapped_region. Like the anonymous condition? Which might explain the problems I was having with it before.

1 0

Review Queue Needs Attention
by Tom Brinkman 11 Dec '09

11 Dec '09

Paul A. Bristow said >> Yes - *exposure* is the key to inspiring authors to refine their contributions. >> I've argued for some time for a 'experimental' or 'candidate' status. >> (And a state not damned with faint praise like 'unstable' - which is perhaps >> better described as 'likely_to_be_improved' rather than actively 'not stable'). >> We'd need a simpler review process to agree to put a library into this >> 'candidate state' from the 'sandbox state'. >> This would mean that when libraries are reviewed for full acceptance, they >> should have received much more use, feedback and refinement, and be really ready >> for use by those who want a 'release quality' stable product - including decent >> documentation. Agreed. Here is the current review queue. 1) Lexer Ben Hanson 2) Shifted Pointer Phil Bouchard 3) Logging John Torjo 4) Join Yigong Liu 5) Pimpl Vladimir Batov 6) Task Oliver Kowalke 7) Endian Beman Dawes 8) Meta State Machine (MSM) Christophe Henry 9) Conversion Vicente Botet 10) Sorting Steven Ross 11) GIL.IO Christian Henning 12) AutoBuffer Thorsten Ottosen 14) Log Andrey Semashev 15) String Convert Vladimir Batov 16) Move Ion Gaztañaga 17) Containers Ion Gaztañaga 18) Interval Containers Joachim Faulhaber 19) Type Traits Extensions Frédéric Bron 20) Interthreads Vicente Botet 21) Bitfield Vicente Botet 22) Lockfree Ivo 23) Faster Signam slots Helge Bahmann The number of review requests is far outstripping the number of volunteer review managers. Considering about six libraries per year are getting reviewed, this is well over a three year back log. And this does not even include some other libraries that have been discussed, but not included. There are alot of good ideas here, they should not be ignored. Some of these libraries have been in the queue for over 18 months. My idea to fix this is well known. I would like something similar to Paul's approach. Those of you who have argued against a "non-stable" branch of boost, how would you propose fixing this "review queue" problem.

11 25

Re: [boost] [MSM] Comparison with ad-hoc FSM implementation
by Christophe Henry 11 Dec '09

11 Dec '09

Hi Phil, >> I fully agree with Michael's point that more complicated constructs >> are really messy to implement with ad-hoc techniques. >Do we have any concrete examples of larger problems anywhere? I would >be interested to see them so that I can consider how I would implement them. Sure. Please have a look in the MSM doc at the MsmSession_Handout.pdf. At the last page, you'll see the description of an iPod nano. This example was presented in the BoostCon hands-on session. We managed to write a fsm representing a full mock-up in 90mn (of course we cheated a bit as I had prepared something). And we immediately generated the code. Do you think it would be possible with an ad-hoc solution in such a short time (or even double more)? >> But on the construct presented by Phil I personally find a lot to criticize. >> First of all, it is a mess. > >I would be much happier if you could please avoid using language like this. Please excuse me but I fail to understand what you mean. Being not a native speaker, I'd greatly appreciate some help as I wasn't under the impression to use a rude language. >> what really bugs me is the mix of fsm structure with >> character processing. > >But in this application, the FSM is driven by the characters that are >received - that is what it is all about. Why do you want to separate >it out into two things? Because I consider that the purely fsm part of the application should be separated from the character analysis. A close look at the functor will show you that it handles character analysis and then generates an event. The fsm then decides what to do with the event. A different fsm can choose to reuse the code in another context. I find this pretty natural. Character analysis on one side, handling on the other. >>>void process_event(const character_received_event& e) { .... >>>void process_event(const sigwinch_event& e) { .... >> >> Having to write overloads in a template word??? No, thanks! > >Can you please explain why ? I find it pretty clear that template <class Event> void process_event(Event const&); constitutes a better interface than (adding more events): void process_event(const character_received_event& e) { .... void process_event(const sigwinch_event& e) { .... void process_event(const some_other_event& e) { .... void process_event(const yet_another_other_event& e) { .... etc. >Consider the transition from the seen_escape state to the normal state, >which occurs at the end of an escape sequence. For this you would need >a last_char_of_escape_sequence event. You would need to consider this >in your case statement above (e.g. letters normally do terminate the >escape sequence but digits normally don't). But this categorisation >happens for received characters whatever state the machine is currently >in. So in the transition table you would need rules like this: > >row < normal, escape_char, seen_escape >, >row < normal, last_char_of_escape_sequence, normal >, >row < normal, not_last_char_of_escape_sequence, normal >, >row < seen_escape, escape_char, seen_escape >, >row < seen_escape, last_char_of_escape_sequence, normal >, >row < seen_escape, not_last_char_of_escape_sequence, seen_escape >, The example seems to grow at will, but the issue stays the same. I operated a seemingly small change, concretely: case escape: state = seen_escape; break; was replaced by a functor containing: case escape: fsm.process_event(escape_char()); break; Which I can repeat any time you set the state variable inside a case statement. The only difference is that now, the code analysing the character has no idea of the fsm structure, which means it can be reused in another fsm. This is made possible by the use of a functor. Otherwise, it's your algorithm, only organized in such a way that you have a better chance of reuse in another fsm context. Whatever, I just wanted to show another way of implementing the same algorithm using MSM, with the same performance and a better chance of reuse. A bigger application will probably illustrate better the strength of MSM. If you look at the iPod example, you'll see how the library will do things which are very hard with an ad-hoc solution. I'd be interested to see how an such implementation would solve this problem. I think it might constitute a more interesting discussion base. Regards, Christophe

2 1

[BoostCon 2010] Submission deadline extended
by Hartmut Kaiser 11 Dec '09

11 Dec '09

In 2010 BoostCon celebrates its 4th anniversary in Aspen. You are invited to submit a paper or a session (see the original call for participation here: http://www.boostcon.com/community/wiki/show/Start/) for BoostCon 2010, Aspen on May 10 - May 14th, 2010. Please, see http://www.boostcon.com for more information. Important dates: Proposal submissions Extended to January 3, 2010 Proposals decisions sent January 25, 2010 (tentative program available) Fully scheduled program available February 10, 2010 Session materials due April 15, 2010 Hartmut Kaiser, email: hartmut.kaiser(a)gmail.com (Program Committee Chair) David Abrahams, email: dave(a)boostpro.com (Conference Chair) On behalf of the conference organizers

1 0

Re: [boost] [MSM] compile errors
by Christophe Henry 11 Dec '09

11 Dec '09

>As we saw over in the "[fusion] change on trunk breaking proto" thread, >even VC10 has bugs that cause random failures in proto's tests. Using >proto isn't necessarily a talisman against compile errors. Sure. In this case, proto has only a very limited role, at most the one of a trigger. Specifically, VC8 has problems with enable_if for template class specializations. Interestingly, it has a problem with Process2_ eventhough it doesn't even use it, just including the header is enough to get the bug. Even more interesting, the next template specialization (Get_Flag_) doesn't seem to disturb the compiler... Anyway, all VC compilers suffer from the same disease. Most template constructs will manage to compile once, but repeat the operation too much and it gives up. I'd bet that Microsoft uses tons of test cases which try any construct once. I actually only try to support eUML for VC8 to honor the fact that it implemented a typeof back in 2005. Christophe

1 0

[MSM] compile errors
by Christophe Henry 11 Dec '09

11 Dec '09

Hi Michael, >I'm trying to compile SimpleWithFunctors.cpp or SimpleTutorialEuml.cpp >with VC8. Using the original review source code package (not the new >eUML syntax). VC8 will drive me crazy. Now it starts complaining about stuff it doesn't use :( I just committed in the sandbox a fix for VC8 (euml/common.hpp). This should now work. But VC8 will not manage much more than this tutorial. As I wrote, it has very limited eUML support. >FWIW. gcc-4.3.3 compiles SimpleTutorial.cpp just fine but has issues with >SimpleTutorialEuml.cpp. Pages and pages of a compiler babble about >boost::fusion::void_ and mpl_::na. I tried only until 1.40. I just checked (with g++4.3 and 1.39, and with g++4.4 and 1.40) and it works without warning, so either we have different warning options (I have the default) or there is a surprise in 1.41. I'll have a look at it. Sorry for the annoyance. Christophe

3 4