Re: [boost] [msm] Review

Subject: Re: [boost] [msm] Review
From: Darryl Green (darryl.green_at_[hidden])
Date: 2009-12-14 09:22:23

On Sun, 2009-12-13 at 23:44 +0100, Christophe Henry wrote:
> >I'm out of time (not just the review period but other time constraints)
> >for doing a proper review, something I regret. However I would like to
> >vote for the acceptance of the msm backend conditional on:
> >
> >Improvements to documentation to formally and fully describe the msm
> >interface.
> 
> As already said, I agree to add a reference.
> 
> >Inclusion of straight-forward out of the box configurability to support
> >large FSMs.
> 
> Unfortunately, there is nothing I can do to improve the compilers :(
> 
> >Yes, with gcc 4.2.4
> >
> >  - Did you have any problems?
> >
> >I couldn't get the euml examples to compile. I'm not sure whether this
> >is an environment problem (I was using boost 1.40 on Debian testing) or
> >something else. I didn't spend much time investigating.
> 
> As stated in the documentation, g++ supports eUML only from 4.3.  It
> is therefore normal that you didn't manage it.
> eUML support is repeated several times in the documentation.

Sorry - I did start testing using 4.3, had some problems and went back
to 4.2 based on the advice it was best for large fsms (using non-euml
interfacce). Forgot this would create problems with euml!

> 
> >My comments have been almost entirely around the back end and the more
> >direct front-ends to it. I personally think euml should be a separate
> >submission/review, once it is more "review ready". There seem to be
> >boost version and compiler version issues in getting even the examples
> >to compile. These are quite possibly outside of msm/euml itselfs
> >control, but if nothing else, they prevent easy review of euml.
> >
> >I'm actually struggling with the above, because I think having euml in
> >trunk, with tests being run, could facilitate resolving some of these
> >issues. Maybe euml could be included as some experimental/example part
> >of msm initially?
> 
> I'm afraid you're extrapolating a little from your own experience
> using the wrong compiler. There were just 2 issues so far. One with
> VC8, which has a (documented) weak support for eUML. This issue has
> been fixed immediately.
> The second is a reported warning on g++, which I was not able to
> reproduce and needs more concrete input from Michael.
> It doesn't seem sufficient for calling eUML not "review ready".
> True, eUML has more limited compiler support and this is documented
> too. It doesn't make it less ready.
> 

I disagree. Even with gcc 4.3 (I just tried) it isn't usable for
"real" (read large) fsms. Also, the fact that I can't use euml on either
of the "production" compilers I use (one MS and one gnu) is a bit of a
problem. Of course, I can't blame you for that. I don't believe euml is
"production ready" due to scalability and outright ability to compile
issues on various "current" compilers (judging by others responses and a
quick try just now with gcc 4.3). I am not reviewing the academic
excellence (you get A++) but the useability.

Your responses come across as though you feel you are under attack.
Please accept that is not my intent. Rather, I'm don't want msm, when it
reaches a wider audience by being in a boost release, being written off
(by potential users) as unusable experimental rubbish due to
difficulties with euml.

Why don't you extend your lib while the compiler vendors work on their
bugs, we all get new hardware etc. Then submit it (the euml support)
when it is more usable? Probably all you have to do is wait - because
there isn't anything obviously wrong with euml - its the inability to
make practical use of it that causes me to reject it (to protect it) for
now.

A review (mini review?) when people are more able to evaluate it, and
when the msm back end has been seen by a wider audience may result in a
better "product". Please understand that I actually really like euml,
(and proto). I want both to succeed!

Of course, thats all just opinion. I could be completely wrong about
these perceived "dangers" of too early release.

> If it's bothering you to have the source/target states as argument,
> it's trivial for me to add a new row type. I wouldn't want to force
> macros on you.

I honestly haven't had time to work on the ideal dispatch interface.
Idon't think there is one. I belive what you have is close to "right"
for the "kernel" and could be wrapped (not necessarily with macros) as
required. Good quality wrappers could be added to the lib if you like...
I don't have a request for anything specific yet....

> 
> >I haven't tried to do it yet, but it struck me that I could potentially
> >use the state types alone (not the object ref parameters) as tags to
> >determine which object to dispatch to. If I chose to have objects that
> >were 1:1 with states I could - but it wouldn't be part of the fsm
> >kernel. Is that concrete enough to make sense?
> 
> It is ;-)
> I'm just unsure if I follow you. With "object to dispatch to", do you
> mean objects in your application? Why not use the state for this? I'm
> interested in your use case for future developments.

Yes I do mean objects in my application. The object model isn't
necessarily tied to states as objects at all. In a protocol stacks I
find layers (and often layers within layers), management interfaces,
hardware interfaces (with different impls for different hardware), user
data flow vs protocol "state" etc to be some of the things that map to
objects.

Of course, this isn't exclusive, I also find use for "activation states"
for which Statechart's construct on state entry model works well.

I was looking for a way to support both from the same kernel.

> >No, I don't agree at all. I can't see how you can argue that having
> >transitions specified in states as statechart does is less clear than a
> >separate transition table, then claim that having enty/exit actions
> >specified in states is more clear than having them similarly separated.
> 
> Good point, which I need to explain. The problem with Statechart's
> approach is that states do the job which should be the state
> machine's. This increases the importance of states in Statechart, at
> the cost of the state machine, which becomes little more than a
> wrapper.

I understand the architectural difference and hence, perhaps your real
point. However on a pure syntax of state machine definition basis I
don't agree because the declarations of the states (in the absence of
those evil custom reactions) can be read as a transition table very
easily. It is simply a transition table that forces all transitions from
a given source state to be grouped together. I find this highly
readable. That doesn't mean I object to the use of a "pure" transition
table notation, which is also highly readable. Once the fsm gets big
enough to start splitting up states into separate headers, etc it isn't
apparent that an enormous transition table is really any better.

But this is all highly subjective and I don't really have a strong
opinion one way or the other on a purely aesthetic or clarity basis.

>  It also mixes entry/exit with the rest of the transition.

I don't see that at all. But I'm also concerned that we are busy
critiquing Statechart, and I don't see that as directly relevant. I'm
trying to understand msm.

> It's not my reading of the philosophy behind the UML specification.
> I think the states have a role, state machines another. State machines
> implement the structure of the diagram. States implement
> state-specific job

Well - arguably they don't have any job except to act as labels for the
vertices - the edges (transitions) need something to connect.

> , which is much less than the state machines'.

What you are calling a state seems to be everything outside of the "is a
vertex" part of what I'd call a state. I'm tempted to say that the
concepts you are describing as state are everything that is
non-essential to the classical concept of a state. I think you need a
strong justification for including such concepts.

>  Their
> jobs are:
> - entry/exit
> - holding state-specific data (not in the UML standard but still
> sounds logical to me)

I see these as incredibly closely related in a C++ FSM model. This is
because, in my experience, the actions done in entry and exit actions
are resource management-like, very similar to the role of constructors
and destructors (or more generally scope entry/exit). I had intuited
that the rational for including entry/exit actions in fsm notations was
to model that scope concept. From a pure execution of the fsm
perspective could just as easily be tacked onto the end/beginning of
every transition action on edges going to/from that state. The point of
the entry/exit action notation is to make sure certain pre/post
conditions are always met regardless of how the state is entered/exited.
It looks like RAII to me. State specific data that outlives state
activation doesn't fit that model terribly well. As you say, it isn't
part of UML. If you like, take this as a request to drop all state
specific data support and provide only actions, including the very
important state entry/exit actions. I propose this because I believe
this is minimal but sufficient to construct FSMs having:

1) No state local storage, but a set of pre and post conditions that
must be assured as part of state activation.

2) State activation scoped state local storage (a variant of the above)

If the problem needs persistent (beyond the scope of state activation)
storage I believe it should be implemented outside of the scope of the
FSM (the overall FSM using class HAS_A FSM). This is consistent with
Harel's approach. I can't say for sure if it matches UML because I
haven't studied its full/formal definition.

Alternatively, if the interface of the class genuinely is that of an
FSM, the FSM HAS_A storage. The storage can be arbitrarily complex, but
I see no requirement that it be composed strictly of objects that
represent states (or is that states that represent objects?).

> - (possibly) internal transitions

Possibly. But one could consider execution of entry/exit actions to be a
property of the transition. And impose a rule that transitions that do
not execute entry/exit actions must have the same state as both source
and destination (i.e. must be internal transitions). I prefer that
approach.

> 
> When you look at a transition table, you see the structure of the
> machine and only the structure.

To the extent that the transitions describe graph edges and the graph is
the structure, yes. Hopefully the states are called something more
informative than S1..Sn and the actions and guards have meaningful names
- or are lambdas. Lets call it labeled structure. I would like to see
the "ring" around each vertex in that structure that enforces pre and
post conditions (ie the entry and exit actions) in that structure.
 
> (EUML shows you more, but this is not a classical state machine front-end :) )

Well - what is classical anyway? Is Harel classical? I think you might
be surprised at the sorts of attributes/actions to do with object
instantiation, association and method invocation it included?
 
> AFAIK you didn't complain about not seeing the actions/guards code in
> the transition table.

No I didn't and I don't. Violent agreement? I don't understand what
point you are trying to make.

> 
> To illustrate the difference of roles, could you please have a look at
> the examples iPodSearch or iPodSearchEuml? This is a good example.

I already have. I didn't find it compelling. But hopefully your detailed
description will be illuminating.

> Take the state StringFind. Its description would be:
> - looks if the string contained in an event is found in the string
> contained in the state.

> - if found, the state sends the event Found on the current state machine.
> - else it sends NotFound.
> 

> Note that StringFind has no idea of which fsm it is used in. It only
> gets a generic fsm as argument of on_entry.

Yes. No problem at all with that.

> The same documentation is possible for the other states (Foreach and Insert).

Good. So far I'm not seeing anything (except the fact that you for some
reason make the StringFind class a state) that conflicts with anything
I've said/proposed.

> 
> So what do we have? 3 states, perfectly reusable and implementing some
> valuable job. And they have no idea of the transition table.

I don't understand why you call 3 classes that have absolutely nothing
special about them beyond the fact that they post events to an FSM
states. There is no need for them to be part of the FSM at all.

> Now imagine that MSM comes in with more states, transform, remove,
> push_back, etc.
> Using a supporting CASE tool, one could easily build complex algorithms.

It is fine to take abstract concepts and map them to states. But
modeling the real world isn't always that neat. It is an unnecessary
(and imho misguided) constraint to require that objects implementing
behavior that is triggered by events be "states" of the FSM. I realise
(using the functor approach) you don't actually impose such a
constraint, but there are traces of this model showing throuigh by virue
of state instantiation and making entry and exit methods of those
instantiated state objects. I think those traces should be removed...

> 
> >I personally find this response vague
> 
> Is this already less vague?

Yes. But unless I am misunderstanding it makes me more convinced that
the state objects are not a necessary or core concept for generalised
fsms.

> About the CASE tool, I showed an early prototype in the last BoostCon
> and plan to start working on it again after the review. I only ask for
> some patience (for the tool, the state library concept and the
> back-ends) because I'm doing all this at night, so I'm not so fast ;-)

Don't let me discourage you. I think what you are doing is great. I just
don't see the parts beyond the backend as being close enough to being
release ready to accept. They are certainly well developed enough to
review (in a general sense of review) and I'm impressed with what I see.
Please don't take my comments re not accepting the rest of the library
yet as anything but a reflection of that distance to go. It is not a
rejection of the work done so far.

> >I'm not outright against state instantiation. I can't be. I do really
> >like statechart's state instantiation model. However, in the review of
> >that library I did ask if there was any way that the costs associated
> >with it could be avoided when not strictly needed (when the state
> >contains no state-local storage). I'm asking a very similar question
> >here, but somewhat more emphatically because this is the Msm (first M is
> >important!) and it should surely be able to do this
> 
> There are limits to what the first M can do though (besides, you know
> what that means to your compiler).
> But I now understand the point and find the idea interesting. It'd
> probably be possible to instantiate only states with local-storage. It
> would require a tag (as I won't be able to rely on sizeof) but it
> could be possible.
> Thanks for the good point!
> 

No problem :-) 

> >> - make a 3rd one, more dynamic, which I want to do since a while.
> >> - make no review until all possible back-ends are done and make
> >> everybody wait at least one more year.
> >>
> >> Do we agree that solution 1 seems the more logical?
> >
> >Logical from what perspective? Buying votes or producing a good library?
> >Politics... ;-)
> 
> It's not politics and even less buying votes, but a simple recognition
> that each developer will use different features and if he doesn't find
> his, he'll simply not use the library.
> I'm pretty sure, if I had only proposed a mini-backend, many would
> have complained about the lack of features, and they'd be right.
> The say 20-40% of UML features which you use won't be the same as what
> another developer will use. So, a library needs to support as much as
> possible.

Believe it or not, I expect to use every UML feature of the library.
None of that use will use state local storage in the sense that it is
implemented by the library though... Seems a bit odd that this is the
only feature I don't expect to use?

> I think MSM is a good library as it offers almost all of UML features
> at very low runtime cost. The only high cost is the compile-time.
> 
> Now it's proven that the concepts of the MPL book were right, that it
> was indeed possible to implement at least as much as statechart with
> metaprogramming techniques.
> That this wasn't evident, as shown here
> (http://lists.boost.org/Archives/boost/2005/03/81853.php).
> I hope you'll trust me that if I can also implement less ;-)
> 
> If I get you right, the only major problem you have with the back-end
> is the automatic, unavoidable instantiation of states? Doesn't sound
> like an impossible problem to me :)

It isn't. I'm merely trying to provide a constructive critique of that
design choice. I hope it has been helpful. Thanks for finding the time
to go into such depth in responding to this.

Darryl.