Re: [boost] What I Hate About How We Abstract System Dependence

From: Mathew Robertson (mathew.robertson_at_[hidden])
Date: 2005-03-07 18:31:58

> > I'd argue that we know exactly - no more, no less -> either the
> > operating system supports a capability, or it doesn't. The only
> > real question is, 'when do we detect the capabilities,
> > at build time? at run time?'
> 
> I think we need more granularity than this.  One important
> counterexample is the case when a capability may be enabled while the
> operating system is running.  For example, a driver may be loaded that
> presents a high-quality random number source.

How can you get more granularity than run time?

> >>Here’s another one of my favorite examples, in Boost.Threads
> >>(“Boost.Threads,” <http://www.boost.org/doc/html/threads.html>).
> >>boost::thread lacks a method to forcefully terminate a thread, despite
> >>the fact that many threading systems have one.
> > 
> > So most people go with an implementation that does not provide forcible termination.
> 
> OK.  I'm not concerned with this particular feature, but the fact that
> there will always be some feature that the system supports that a user
> may want that Boost.Threads will not implement.  If its not this, then
> it might be thread priority, and if not that, then it might be setting a
> thread's processor affinity, or who knows what else.

Setting the processor affinity on FreeBSD 4.x (non-existant setAfinity function) or Linux 2.4 (returns Esomthing) doesn't work.

> My point is that the concrete Boost.Thread is not open for extension.
> If you want a feature it doesn't provide, you either have to modify the
> sources, or just write yourself an entirely new thread ibrary.  This is
> should not be the way we do it in C++.

This is exactly the point of C++ -> it provides you a way to extend existing code through polymorphism and/or encapsulation.  If those techniques dont work, you can simply a) implement you own code, b) inport third-party code.

It appears that Boost.Thread doesn't provide the functionality that you need - which may be a reasonable claim (I personally like the ZThread library).  In which case, then I'd suggest you offer up an alternative for the implementation rather than saying something about the magic of polymorphism, as most people on the list are well aware of the concept.

> >>Besides the inability of our concrete interfaces to support a capability
> >>set that varies,
> > 
> > The Boost concrete interfaces only provide non-varying implementations
> > so that the code is predictable on all supported platforms.  Would you
> > prefer that you would need to test your code on every revision of
> > every platform?
> 
> With the polymorphic scheme I propose, if you only want the least common
> denominator interface, then thats all you have to use, and there is no
> changes needed to your testing procedure.
>
> Furthermore, static, compile-time checking can still be achieved by
> using concrete factories.  For example, if you'd like compilation to
> fail if POSIX facilities are unavailible, code like this could be written.
> 
>   Process p = POSIXProcessFactory::getProcess("some_task");
>   p->getProcPath();

... and how is this portable to win32 / VMS / etc, without writing #define's everywhere?
 
> > Its current form is a result of understanding the fundamental problems:
> > - single reasonable useful API
> > - an API that works across most platforms
> > - an implementation that is predictable across the supported platforms
> 
> I am not proposing altering these guarantees--and for what it's worth, I
> agree these are very important characteristics of Boost libraries.
> 
> However, I am proposing that in addition to providing this basic,
> portable, guaranteed interface, we allow the user to 'upcast' to
> system-specific interfaces, with compile-time or runtime checking, at
> their option.

ok - that wasn't clear, and FWIW I tend to agree on that point, ie: the programmer should be able to get access to the native capabilities if they should choose to do so.  However I'm not sure that I would want to see POSIXProcessFactory in the API...  However, getting access to the OS-specific handle would be useful.

> > The proc filesystem is a directory structure - not a POSIX definition - the comparison is invalid.
> > 
> > In the example of a debugger, what does the debuggin process do if there is no debugging capabilities for that platform?
> > Again the problem is "at what point do we detect the available capabilities".
> 
> Presumably the factory/copy operation fails, and the debugger has to
> cope with this situation.  This is unavoidable in any case, regardless
> of what abstraction is being used, as it is impossible to determine if
> /proc is mounted at compile-time.

Of course... but I would expect the Boost API to be along the lines of:

Debugger d = DebuggerFactory::getInstance()

not:

Debugger d = SlashDevDebuggerFactory::getInstance()

> >>I’m calling for polymorphism.  These interfaces really are conceptually
> >>polymorphic; let’s reflect that in our language.  Let’s give the user
> >>the tools she needs to be able to write her own compatible classes when
> >>the ones we write prove insufficient.
> > 
> > I'd argue that the capabilities arn't polymorphic. ie you simply couldn't do somthing like this:
> >
> > class Process {
> >    virtual void run();
> > }
> >
> 
> Let me re-arrange your example slightly.
> 
> class Process_with_slash_proc : virtual public Process {
>   // ...
> };
> 
>   Process p = ProcessFactory::getProcess("some_task");
>   try {
>     Process_with_slash_proc p2 = Clone_process_with_slash_proc(p);
>     p2->getProcPath();
>   }
>   catch(something &) {
>   // ...
>   }
> 
> This sort of runtime detection would presumably by a rare case, but its
> an important case!  This system also allows us to do things like load a
> DLL at runtime that knows how to perform some sort of operation on a
> Process that the user, or the original author of Process, doesn't even
> know about.

I'd like to mention one point:

>   Process p = ProcessFactory::getProcess("some_task");

contradicts your example of: 

>   Process p = POSIXProcessFactory::getProcess("some_task");

I think I understand what you are trying to say, something along the lines of "lets use factory methods for generating the instances, so that, at run time, we get the most capabilites available from the OS" - which I throughly agree with - I'm just not sure that you have considered some of the examples.

> A more common case:
> 
> class signalable_process : virtual public Process {
>   public:
>     int send_signal(int);
> };
> 
> class POSIX_process : public signalable_process /* , ... */ {
>   // ...
> };
> 
>   POSIX_Process p = POSIXProcessFactory::getProcess("some_task");
>   p.send_signal(42);
> 
> Again, this is just an naïve example.  While this isn't identical to a
> real design, it shows how we can reflect capability sets through
> inheritance.  One could imagine having an HPUX class that derived from
> the POSIX class that presented additional HPUX-specific interfaces.

not a valid example - as signals are pretty much useless on win32.  In real code you would most likely need to write:

Process p = ProcessFactory::getProcess("some_task");
#ifdef POSIX
try {
   POSIXProcess pp = dynamic_cast<POSIXProcess>(p);
   p.send_signal(42);
} catch (...) {...}
#else // WIN32
try {
   WIN32Process pp = dynamic_cast<WIN32Process>(p);
   // dont deliver signal as this makes no sense on win32
} catch (...) {...}
#endif

Now WIN32Process may implement an empty stub for send_signal(), but I'd argue that that idea is flawed.
 
> > So that you essentially get objects which will have the same
> > generic / portable' API, but you also get the OS specific
> > functionality.
> 
> Exactly!  This is exactly what I'm getting at.
> 
> > Then the programmer decides to use that OS specific functionality.
> > The end result?  Non-portable code.
> 
> The programmer deliberately chose to use this non-portable
> functionality, and presumably he knew exactly what he was doing.  C++'s
> type safety leaves very little room for accidents.

agreed - but the actual implementation would most likely need to use dynamic casts.

> > Why doesn't the programmer simply just the use the native API...? Why
> > bother using Boost at all?
> 
> So he can reuse the Boost code.  For example, the process class already
> does 95% of what he needs.  He just needs one other feature it's missing.
> 
> To me, this is the essence of re-use, and the open-closed principle, and
> what C++ is supposed to be.
> 
> It's very odd to me that you think it would be better to refuse to allow
> the programmer to trivially extend the class, and instead force him to
> reimplement the entire process library.

I never said that.  And I never said that Boost.Thread should not be extensible.

What I said was that certain design decisions were made - if you dont like them, show us the working code.  Specifically I am asking that you dont rant, but to explain deficiencies so that they can but fixed. eg:

"Hi folks,  I have been using Boost.Thread for a while now... I think its API doesn't suite being used as a base class, and here is an example of where it fails... [some example / test-able code]   Am I using the API correctly, or is there a deficancy?  What do you think?"

> Besides that, the ability to extend in this manner leads to a common
> framework.  Unlike the bad kind of framework that tends to lock people
> out more than anythin else, this kind of framework provides common
> ground on which people may make separate, but compatible, extensions.
> For instance, Katie might implement a class that sends signals to a
> process, and Amy might implement a class that waits for process
> termination, and Nicole could use both of these capabilities together,
> on the same process, at the same time.  This is reuse.
> 
> > I dont understand... doesn't:
> > 
> > "A more significant concern is the additional machinery needed to support virtual inheritance."
> > 
> > conflict with:
> > 
> > "I’m not at all worried about making operations that would be normal function calls into virtual function calls..."
> 
> In the former quote, I'm talking about the cost associated with doing
> upcasts with dynamic_cast, and similar operations.  In the latter, I'm
> talking about the simple indirect calls.
> 
> For someone who only chooses to use the least common denominator
> functionality, they will only need to be concerned with the latter sort
> overhead.

> >> Would you use a library such as Boost.Thread if it had been rewritten in this manner?
> > 
> > No - it is not deterministic.
> 
> I don't understand your objection.  If you choose to use only the least
> common denominator functionality, I'd expect the behavior to be,
> formally and practically, identical to the status quo concrete class.

Nope, not formally, only practically - by making the class polymorphic, you have just added a VTable lookup to function calls (compared with no VTable lookup previously).  That said, a VTable lookup wont affect most people.

Note that the "No - its not deterministic" refers to not knowing what API a given object will have, without doing dynamic casts (not to the relative number of CPU cycles needed to invoke virtual functions, etc).

> However, for those users who need features only availible in some
> environments, they are free to re-use the existing code, plus add their
> own code to support their particular use-case.
> 
> I can't beleive that it is better to *not* give a user a chance at
> extending the class.

Again, I never said that.

> Thank you for the detailed analysis and criticism,

No problem,
Mathew