Re: [boost] [Boost-users] [context review] Review starts today, Mars 21st 2011

Subject: Re: [boost] [Boost-users] [context review] Review starts today, Mars 21st 2011
From: Oliver Kowalke (oliver.kowalke_at_[hidden])
Date: 2011-03-23 04:07:11

> I see that Boost.Fiber is on the review queue, but Boost.Tasklet and
> Boost.Task are not yet?

boost.tasklet will be renamed to boost.strand (see my git repo). because all of those lib require boost.context and hafe to be reqorked (except boost.fiber) the review wizards requested to remove boost.task from the review queue.
If boost.context is accepted and in a final state I'll request a review for the other libs. 
 

> > Boost.Context allows to select the mechanism provided by the operating
> system (ucontext on UNIX, Windows Fibers on Windows) or a faster
> implementation provided by this library (fcontext using assembler).
> 
> Note on the alternative implementation: I am not enthused about the
> idea of embedding the fcontext decision in source code. I would rather
> make that decision externally in the build machinery.

I've pulished an alternative version of boost.context - the user can decide in its code which implemention should be used.

> void (fn)(void*) and void* vp are very classic-C. While I'm sure there
> are those who prefer to work at that level, I'd also want overloads
> supporting a templatized nullary callable, like Fiber, plus the
> args-for-implicit-bind variant. Hopefully Fiber and other high-level
> abstractions implemented on Context could simply leverage Context's
> support rather than reimplementing it. Since every library built on
> top of Context will want to support those constructs, if that support
> can be centralized in Context, it should be.

Becuae most of the work is done ins assembler I'd like to keep the things simple as possible. Supporting templatized ctors + args-for-implicit-bind variant would make the assembler too much complicated and thus error prone (different C++ calling conventions on the platforms etc.).
So I want to stuck with the C-style callback and track boost.context as a building block for other libs providing higher abstractions.

 
> I'm not sure I understand the usefulness of a context object not
> currently associated with a stack object, but I realize that
> corresponds with a state designed into the underlying machinery, e.g.
> ucontext.h. Oh: the initial current context doesn't manage an explicit
> stack. Maybe a base class that represents a context with implicit
> stack, and a subclass that represents a context with explicit stack?

You have to differentiate between a context created to execute a new execution context (the ctor accepting funtion, stack etc.) which manages its own stack (passed as argument) and the context instance grapping the current execution context (default ctor) which collect the CPU state but does not manage the current stack because it its already managedd by the operating system (in the case of main()-function for instance) or by another context create by its ctor with protected_stack instance as argument.

void main()
{
  ...
  // ctx1 grapps the current CPU state
  // but does not manage a stack because the 
  // current stack belongs to the operating system/process
  context ctx1; 
  
  // ctx2 represent a new execution path executing my_fn()
  // it manages its own stack
  context ctx2( my_fn, 0, protected_stack( 65536) );

  // now we jump from the curretn execution context ctx1
  // to the new execution context ctx2 which will execute
  // function my_fn()
  ctx1.jump_to( ctx2);

  // we return from my_fn() == jumped out of execution context ctx2
  // back to ctx1
  ...
}

ctx1 pin-points (you could have mutliple) a specific point in the code/execution path. 

> (Is the static variable that underlies boost::context::current()
> thread-local?)

context has no static variables and no boost::context::current() (maybe you are using an old version?).

> Fiber-local-storage on Windows: may I suggest a context-local<T>
> abstraction for this library, implemented as Fiber-local for the
> Windows Fibers implementation and thread-local elsewhere?

hmm - this what I want to handle in libs with higher abstractions. So boost.context can be thin as possible.

> Exceptions: "must not throw exceptions" may be too limiting. It's hard
> for a typical coder to guarantee that no underlying code will throw.
> Of course you could require that every fn passed to context's
> constructor must have an explicit outermost try/catch(...) -- but
> that's the sort of boilerplate that's easy to forget. context could
> provide a wrapper containing that try/catch(...). In fact, you could
> even use Boost.Exception to propagate any such exception to "somewhere
> else" -- perhaps a context user's policy decision? Then one such
> policy choice could be "I guarantee no underlying code will throw,"
> for those anxious to avoid the overhead.

The function which is pased as first argument to the ctor of context must not throw. The reason is that it is passed to C-functions (== functions declared as extern "C") and throwing an exception through C-code results in undefined behaviour.

> Exception management may feel like a topic best left for a
> higher-level abstraction -- but I don't see how that could work. If
> the higher-level abstraction is implemented in terms of Context, and
> if Context flatly forbids exceptions, how could the higher layer
> support them?

Exception hamdling should be done by higher abstractions. For instance using packaged_task/future (as shown in boost.tasklet).

> On the other hand, if it *is* reasonable for Context to defer
> exception support to a higher level, then the documentation should be
> revised to state the requirements for that support.

I believe yes.

> What's the use of a stack object representing not-a-stack? Wouldn't
> the abstraction be simpler and more robust if every stack object is
> guaranteed to represent a valid stack? That would seem to simplify
> some of the semantics of Context as well.

stack must support move-semantics -> a moved stack must be represented == not-a-stack.
This is usefull for reducing memory allocation/deallocation for stacks.
boost.tasklet reuses (== moves) stacks from a context which is finished and caches the stack. If a new context has to created in moves the recycled stack into the new context. That's trhre reason why stack must be move-able.

> That said -- I'm a bit concerned that the documentation seems to imply
> that stacks always grow downwards. Are we really never going to
> encounter machines on which the opposite is true? Shouldn't the stack
> concept include some support for directionality?

On all platforms I've been working on the stack grows downwards. 

> The stack concept is fairly dense. Would it make any sense to define
> an actual stack abstract base class? Because Context is
> template-based, an implementation can (and protected_stack should)
> avoid that base class -- but it could provide user stack
> implementations with a starting point.

because the stack is a template argument of context it represents an implicit interface - so no explicit interface like abstract_stack is required. The current solution doesn't require stakc to derive from a base class.

> Thumbs up on providing protected_stack rather than a plain memory area.

protected_stack appends a guard page at the end of the stack. This guard page issues sefault/access violation if you acces the address of that page.
This is usefull if the size of the stack is too small. Ypou could use your own stack implementation without such a guard page. But if you excced the stack range you overwrite in the worst case the memory of your own application.

> Again, the need to account for a context object that isn't currently
> managing a stack feels like clutter.

see above - pin-point of current execution path/context

> The Boost.Coroutines documentation contains a discussion of the
> possible unreliability of the GetCurrentFiber() == 0x1E00 trick, and
> the value of calling IsThreadAFiber() on systems that support it. I
> note some ambivalence in context_fiber.hpp about that. Why is the
> IsThreadAFiber() implementation commented out?

That is because of the fact that Windows XP + Sevice Pack 2/3 defines _WIN32_WINNT >= _WIN32_WINNT_VISTA but doesn't declare ::IsThreadAFiber().
I didn't found a way to detect XP + SP 2/3 beside of _WIN32_WINNT.

> > - What is your evaluation of the documentation?
> 
> At first I was confused about the usefulness of release_stack().

moves the stack out of the context instance for inspection or recycling

> The Boost.Coroutines documentation notes that you can save the cost of
> ConvertThreadToFiber() and ConvertFiberToThread() per round trip by
> explicitly calling ConvertThreadToFiber() beforehand.
> context_fiber.hpp is implemented the same way; presumably the same
> note could be useful. Or are you assuming that all cycle-conscious
> users will select the fcontext implementation instead of the Fiber
> implementation?

Holger Grund has some concerns regarding exception handling in the assembler implementation of fcontext on Windows. So I've dissabled it - maybe if I could fix it I'll make it available too.

> There's a confusing phrase in the description of stack::address():
> "Function void* address() must return the address where the stack
> begins to grow downwards (to lower addresses)" -- so far so good, with
> a nod to the note above about stack direction -- "and the stack must
> be moveable."

see my comments above

> This seems to say that the Context library might move the block of
> memory in which the processor allocates stack frames. But that would
> completely invalidate data residing in those stack frames! I *think*
> the phrase is intended to mean that any class implementing the stack
> concept (the object containing stack pointer and size) must be
> moveable. Either way, the intent should be expressed more clearly.
> What is that phrase doing attached to the description of
> stack::address(), anyway?

do refere to moving the stack out of a context? then this don for reusing that memory junk - otherwise the stack get deallocated.

 
> I intend to try replacing that Boost.Coroutines Pth implementation
> with a Context implementation. (Those who still care about OS X 10.4
> could then provide a Context implementation based on Pth. However, I
> myself no longer care.) If Context is accepted into Boost, It strikes
> me that a version of Boost.Coroutines based on Context would be a
> useful evolutionary step in the Vault.

I'm interessted in porting boost.context to MAC OS X (some first stepps in Jamfiles -> allow ucontext) - unforutnately I've no box running MAC OS X. I hope we can work together.

best regards,
Oliver

-- 
GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit 
gratis Handy-Flat! http://portal.gmx.net/de/go/dsl