Re: [boost] What I Hate About How We Abstract System Dependence

From: Mathew Robertson (mathew.robertson_at_[hidden])
Date: 2005-03-07 01:59:59

[snip]
 
> Let me get straight to the heart of the problem: we love our concrete
> interfaces.  Every programmer loves the ability to be able to manipulate
> a class, knowing its complete interface, exactly what sort of animal it
> is, and exactly what its going to do—and know all of this before the
> program ever runs for the first time.

Thats because it makes design and testing easier (ie more predictable), as we are capable of envisioning the implementation...

> My core belief is this.  We don’t fully know the capabilities of the
> system components we manipulate, not at compile time, not at load time,
> and perhaps not even by the time the program halts.  Any interface that
> pretends to is a lie, and most of the time, a pretty bad liar at that.

I'd argue that we know exactly - no more, no less -> either the operating system supports a capability, or it doesn't.  The only real question is, 'when do we detect the capabilities, at build time? at run time?'
 
> Here’s another one of my favorite examples, in Boost.Threads
> (“Boost.Threads,” <http://www.boost.org/doc/html/threads.html>).
> boost::thread lacks a method to forcefully terminate a thread, despite
> the fact that many threading systems have one.

Thats because most forcible terminations result in undefined behaviour.  Think:

"If I have locked a mutex in a worker thread, then I forcibly kill that thread, what state is the mutex in?"

So most people go with an implementation that does not provide forcible termination.

> However, we can’t add
> one, because there exists at least one threading system that doesn’t
> have this feature.  Well, we *could*, but then we’d be playing a game of
> chicken with the user daring them to call a function that has completely
> undecidable behavior.  Now let’s say the user was really determined get
> this feature, and so she decided to write her own thread class.  Nope,
> she loses again!  Because her class is not named boost::thread, the
> class is incompatible with all of the rest of the thread manipulation
> functions, and so is entirely unusable with Boost.Threads.

Boost provides that user with predictable results - forcible termination often results in non-predictable behaviour.

> Do you like scary movies?  I have an idea for one.  It’s about a future
> C++ standard that includes a threading library that I can’t use if I
> want forced termination semantics, or any other feature that any
> operating system has that the library lacks.

This already happens in the present -> compare one *nix to another, even to various flavours of win32...

> Besides the inability of our concrete interfaces to support a capability
> set that varies,

The Boost concrete interfaces only provide non-varying implementations so that the code is predictable on all supported platforms.  Would you prefer that you would need to test your code on every revision of every platform?

> and the lack of ability to reimplement these interfaces
> compatibly, concrete schemes also can’t represent what I call
> /multi-interface systems/.  Some environments, such as Cygwin (“Cygwin
> Information and Installation,” <http://www.cygwin.com>), have two (or
> more) entirely distinct sets of system interfaces that may be used as
> underlying primitives for a particular concept.  In the case of Cygwin
> and threads, the environment supports both the POSIX and Windows thread
> interfaces.  Both may be used simultaneously, and as each system has its
> own unique characteristics, it may be entirely reasonable to do so.
> But, Boost.Threads can’t do this, and really couldn’t possibly be
> expected to, in its current form.

Its current form is a result of understanding the fundamental problems:
- single reasonable useful API
- an API that works across most platforms
- an implementation that is predictable across the supported platforms

> Note that the total set of capabilities is not decidable at compile
> time, or even load time.  For instance, let’s say we’re using a process
> class on a System V-style system.  We’re implementing a debugger, and so
> we’d like to get access to the process’ core memory through the /proc
> interface.  However, up until we actually try to do so, we really have
> no way of knowing whether this is supported, as the /proc filesystem may
> well not be mounted.

The proc filesystem is a directory structure - not a POSIX definition - the comparison is invalid.

In the example of a debugger, what does the debuggin process do if there is no debugging capabilities for that platform?
Again the problem is "at what point do we detect the available capabilities".
 
> I’m calling for polymorphism.  These interfaces really are conceptually
> polymorphic; let’s reflect that in our language.  Let’s give the user
> the tools she needs to be able to write her own compatible classes when
> the ones we write prove insufficient.

I'd argue that the capabilities arn't polymorphic. ie you simply couldn't do somthing like this:

class Process {
   virtual void run();
}

class HPUX_with_slash_proc : public Process {
   std::string getProcPath();
}

class HPUX_without_slash_proc : public Process {
}

//Now lets try to use them:

  Process p = ProcessFactory::getProcess("some_task");
  p->getProcPath();

Now what is meant to happen?

> For the sake of exposition, let me propose a sketch of a possible design
> for a process class.  A generic process is represented by an abstract
> base class.  Derived from it are classes for the major types of
> processes: POSIX, Windows, DCE, and whatever else.  Derived from each of
> those are specific variants of these types, with additional or extended
> capabilities.  For example, a child of the POSIX class might be a class
> implementing the ptrace() process debugging interface.
> 
> Naturally, we’d instantiate objects of these classes with some factory.

So that you essentially get objects which will have the same 'generic / portable' API, but you also get the OS specific functionality.  Then the programmer decides to use that OS specific functionality.  The end result?  Non-portable code.  Why doesn't the programmer simply just the use the native API...? Why bother using Boost at all?

>  Since we might not know at creation time exactly what capabilities the
> system has, or what capabilities are needed, we need a copying mechanism
> to construct a new process from an old one.  This copy might, for
> example, copy the POSIX base, so as to get the process identifier, and
> slice the rest off, as unneeded.  Clearly there would need to be
> significant design effort put into this area.  This and other
> implementation issues are mostly tangential from my primary concerns.

The reality of programming for multiple environments is that "implementation issues" are usually the prime reason why, as your threading example highlights, API's tend to be for the lowest common denominator...

> Lastly, performance needs to be addressed.  I’m not at all worried about
> making operations that would be normal function calls into virtual
> function calls, and you shouldn’t be, either.  An indirect call will
> often have a cost an order of magnitude less than the cost of the actual
> underlying system operation, which might involve many more indirect
> calls, context switches, and synchronization.  A more significant
> concern is the additional machinery needed to support virtual
> inheritance.

I dont understand... doesn't:

"A more significant concern is the additional machinery needed to support virtual inheritance."

conflict with:

"I’m not at all worried about making operations that would be normal function calls into virtual function calls..."

>  RTTI may also be necessary to fully exercise the class
> hierarchy’s capabilities.  However, when using the subset of features
> that would be available to an equivalent concrete implementation, RTTI
> and similar should not be needed, so a user shouldn’t have to pay (too
> much) for what she’s not using.
> 
> So here’s my question to the Boost community.

> How many people have similar concerns and experiences?

Probably everybody...

>  How often, in real code, do concrete classes prove insufficient?

All the time...

> Who here has to entirely reimplement libraries like Boost.Threads for relatively silly reasons?

The Boost.Threads problem you have mentioned above is not silly - if you have a solution for point that I highlighted, please let us know as we would be extremely interested in the implementation.

> Are there any alternate solutions for the problem I describe?
> What unforeseen problems might there be with this polymorphic style?

See above.

Also, what you are describing appears the be a style of that used in the late 80's and early 90's... nice and deep class hierachies (which we now know through experience that it was not such a good fit for a lot of designs...).

>  Would you use a library such as Boost.Thread if it had been rewritten in this manner?

No - it is not deterministic.

I apologise if my response appears head-string - its not meant to be...  If you have specific solutions please explain them.

regards
Mathew