- Boost - lists.boost.org

[context review] Review - alternative version available
by Oliver Kowalke 23 Mar '11

23 Mar '11

Hi, I've uploaded an alternative version to boost vault (boost.context-alternative_version.zip : http://tinyurl.com/6fpnkro) The documentation is available at http://ok73.ok.funpic.de/alternative/boost/libs/context/doc/html/. changes: - fixed bug for x86_64/sysv/elf assembler file: missing assembler directive .type added - as suggested by Artyom and Vicente: - assembler implementation and native implementation of context switching functions compiled and applied as template arg to context<> -> context<> == context< protected_stack, asm_impl > on UNIX, context< protected_stack, native_impl > on Windows - for Windows assembler implementation is disabled Oliver

3 11

Re: [boost] [Boost-users] [context review] Review starts today, Mars 21st 2011
by Oliver Kowalke 23 Mar '11

23 Mar '11

> I see that Boost.Fiber is on the review queue, but Boost.Tasklet and > Boost.Task are not yet? boost.tasklet will be renamed to boost.strand (see my git repo). because all of those lib require boost.context and hafe to be reqorked (except boost.fiber) the review wizards requested to remove boost.task from the review queue. If boost.context is accepted and in a final state I'll request a review for the other libs. > > Boost.Context allows to select the mechanism provided by the operating > system (ucontext on UNIX, Windows Fibers on Windows) or a faster > implementation provided by this library (fcontext using assembler). > > Note on the alternative implementation: I am not enthused about the > idea of embedding the fcontext decision in source code. I would rather > make that decision externally in the build machinery. I've pulished an alternative version of boost.context - the user can decide in its code which implemention should be used. > void (fn)(void*) and void* vp are very classic-C. While I'm sure there > are those who prefer to work at that level, I'd also want overloads > supporting a templatized nullary callable, like Fiber, plus the > args-for-implicit-bind variant. Hopefully Fiber and other high-level > abstractions implemented on Context could simply leverage Context's > support rather than reimplementing it. Since every library built on > top of Context will want to support those constructs, if that support > can be centralized in Context, it should be. Becuae most of the work is done ins assembler I'd like to keep the things simple as possible. Supporting templatized ctors + args-for-implicit-bind variant would make the assembler too much complicated and thus error prone (different C++ calling conventions on the platforms etc.). So I want to stuck with the C-style callback and track boost.context as a building block for other libs providing higher abstractions. > I'm not sure I understand the usefulness of a context object not > currently associated with a stack object, but I realize that > corresponds with a state designed into the underlying machinery, e.g. > ucontext.h. Oh: the initial current context doesn't manage an explicit > stack. Maybe a base class that represents a context with implicit > stack, and a subclass that represents a context with explicit stack? You have to differentiate between a context created to execute a new execution context (the ctor accepting funtion, stack etc.) which manages its own stack (passed as argument) and the context instance grapping the current execution context (default ctor) which collect the CPU state but does not manage the current stack because it its already managedd by the operating system (in the case of main()-function for instance) or by another context create by its ctor with protected_stack instance as argument. void main() { ... // ctx1 grapps the current CPU state // but does not manage a stack because the // current stack belongs to the operating system/process context ctx1; // ctx2 represent a new execution path executing my_fn() // it manages its own stack context ctx2( my_fn, 0, protected_stack( 65536) ); // now we jump from the curretn execution context ctx1 // to the new execution context ctx2 which will execute // function my_fn() ctx1.jump_to( ctx2); // we return from my_fn() == jumped out of execution context ctx2 // back to ctx1 ... } ctx1 pin-points (you could have mutliple) a specific point in the code/execution path. > (Is the static variable that underlies boost::context::current() > thread-local?) context has no static variables and no boost::context::current() (maybe you are using an old version?). > Fiber-local-storage on Windows: may I suggest a context-local<T> > abstraction for this library, implemented as Fiber-local for the > Windows Fibers implementation and thread-local elsewhere? hmm - this what I want to handle in libs with higher abstractions. So boost.context can be thin as possible. > Exceptions: "must not throw exceptions" may be too limiting. It's hard > for a typical coder to guarantee that no underlying code will throw. > Of course you could require that every fn passed to context's > constructor must have an explicit outermost try/catch(...) -- but > that's the sort of boilerplate that's easy to forget. context could > provide a wrapper containing that try/catch(...). In fact, you could > even use Boost.Exception to propagate any such exception to "somewhere > else" -- perhaps a context user's policy decision? Then one such > policy choice could be "I guarantee no underlying code will throw," > for those anxious to avoid the overhead. The function which is pased as first argument to the ctor of context must not throw. The reason is that it is passed to C-functions (== functions declared as extern "C") and throwing an exception through C-code results in undefined behaviour. > Exception management may feel like a topic best left for a > higher-level abstraction -- but I don't see how that could work. If > the higher-level abstraction is implemented in terms of Context, and > if Context flatly forbids exceptions, how could the higher layer > support them? Exception hamdling should be done by higher abstractions. For instance using packaged_task/future (as shown in boost.tasklet). > On the other hand, if it *is* reasonable for Context to defer > exception support to a higher level, then the documentation should be > revised to state the requirements for that support. I believe yes. > What's the use of a stack object representing not-a-stack? Wouldn't > the abstraction be simpler and more robust if every stack object is > guaranteed to represent a valid stack? That would seem to simplify > some of the semantics of Context as well. stack must support move-semantics -> a moved stack must be represented == not-a-stack. This is usefull for reducing memory allocation/deallocation for stacks. boost.tasklet reuses (== moves) stacks from a context which is finished and caches the stack. If a new context has to created in moves the recycled stack into the new context. That's trhre reason why stack must be move-able. > That said -- I'm a bit concerned that the documentation seems to imply > that stacks always grow downwards. Are we really never going to > encounter machines on which the opposite is true? Shouldn't the stack > concept include some support for directionality? On all platforms I've been working on the stack grows downwards. > The stack concept is fairly dense. Would it make any sense to define > an actual stack abstract base class? Because Context is > template-based, an implementation can (and protected_stack should) > avoid that base class -- but it could provide user stack > implementations with a starting point. because the stack is a template argument of context it represents an implicit interface - so no explicit interface like abstract_stack is required. The current solution doesn't require stakc to derive from a base class. > Thumbs up on providing protected_stack rather than a plain memory area. protected_stack appends a guard page at the end of the stack. This guard page issues sefault/access violation if you acces the address of that page. This is usefull if the size of the stack is too small. Ypou could use your own stack implementation without such a guard page. But if you excced the stack range you overwrite in the worst case the memory of your own application. > Again, the need to account for a context object that isn't currently > managing a stack feels like clutter. see above - pin-point of current execution path/context > The Boost.Coroutines documentation contains a discussion of the > possible unreliability of the GetCurrentFiber() == 0x1E00 trick, and > the value of calling IsThreadAFiber() on systems that support it. I > note some ambivalence in context_fiber.hpp about that. Why is the > IsThreadAFiber() implementation commented out? That is because of the fact that Windows XP + Sevice Pack 2/3 defines _WIN32_WINNT >= _WIN32_WINNT_VISTA but doesn't declare ::IsThreadAFiber(). I didn't found a way to detect XP + SP 2/3 beside of _WIN32_WINNT. > > - What is your evaluation of the documentation? > > At first I was confused about the usefulness of release_stack(). moves the stack out of the context instance for inspection or recycling > The Boost.Coroutines documentation notes that you can save the cost of > ConvertThreadToFiber() and ConvertFiberToThread() per round trip by > explicitly calling ConvertThreadToFiber() beforehand. > context_fiber.hpp is implemented the same way; presumably the same > note could be useful. Or are you assuming that all cycle-conscious > users will select the fcontext implementation instead of the Fiber > implementation? Holger Grund has some concerns regarding exception handling in the assembler implementation of fcontext on Windows. So I've dissabled it - maybe if I could fix it I'll make it available too. > There's a confusing phrase in the description of stack::address(): > "Function void* address() must return the address where the stack > begins to grow downwards (to lower addresses)" -- so far so good, with > a nod to the note above about stack direction -- "and the stack must > be moveable." see my comments above > This seems to say that the Context library might move the block of > memory in which the processor allocates stack frames. But that would > completely invalidate data residing in those stack frames! I *think* > the phrase is intended to mean that any class implementing the stack > concept (the object containing stack pointer and size) must be > moveable. Either way, the intent should be expressed more clearly. > What is that phrase doing attached to the description of > stack::address(), anyway? do refere to moving the stack out of a context? then this don for reusing that memory junk - otherwise the stack get deallocated. > I intend to try replacing that Boost.Coroutines Pth implementation > with a Context implementation. (Those who still care about OS X 10.4 > could then provide a Context implementation based on Pth. However, I > myself no longer care.) If Context is accepted into Boost, It strikes > me that a version of Boost.Coroutines based on Context would be a > useful evolutionary step in the Vault. I'm interessted in porting boost.context to MAC OS X (some first stepps in Jamfiles -> allow ucontext) - unforutnately I've no box running MAC OS X. I hope we can work together. best regards, Oliver -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl

2 1

[GSoC11] Checks & Hashes
by Mateusz Nowak 23 Mar '11

23 Mar '11

Dear Paul A. Bristow, I'm 3rd year Informatics student at ETI, Gdańsk University of Technology and I'm highly interested in participating in GSoC 2011. I choose boost and "Checks & Hashes" project idea, because it meets with my coding abilities and "always try to make something useful" general rule. I have 7 years experience in programming and algorithms. I would rate my knowledge: c++ 4/5, svn 3-3.5/5. Some questions about the project. As CRC class seems to be only separate hashcodes library in boost, should the rest of algorithms libraries at least try to inherit crc methods naming and parameters convention or, if decision to use completely different convention, the crc library should be adapted or left as it is now? Actual naming idea is that function name should start with algorithm type - if it produces checksum or hash, than algorithm name (sha, crc, ...) and then variant (like in sha: sha1, sha256,...), so sha1 function name will look like h_sha_1, or for "Simple modulo 256 etc check values and digits." it will be c_simple_256. How many algorithms you expect to be found/implemented, documented, tested and got to work more or less? This would help me making work plan. Also i will have exams between 8-21 June, so my commitment to the project at that time will be low. I hope that this won't be a problem. Regards, Mateusz Nowak.

2 1

Boost related topics proposed by the CGAL Project at the Google Summer of Code 2011
by Andreas Fabri 23 Mar '11

23 Mar '11

Hello, The focus of the CGAL Project is computational geometry, and we use boost wherever it makes sense. We are mentoring organization at the GSoC 2011, and we are looking for students who master boost and who are interested in replacing home-grown solutions by boost solutions in the CGAL library. We propose two topics: Replacing some Basic Non-Geometric Tools in CGAL by Boost/TR1/C++0x Functionality http://www.cgal.org/project_ideas.html#boostification Switching from CGAL::HalfedgeGraph to boost::PlanarEmbedding http://www.cgal.org/project_ideas.html#embeddedGraph Best regards, Andreas Fabri

1 0

Library format required for inclusion in Boost
by Mathias Gaunard 23 Mar '11

23 Mar '11

Hi, I'd like to ask a few questions about what libraries that wish to be included into Boost must look like, and I would like to know whether libraries need to fulfill those requirements for the review or can do so later. Build system: integration with Boost.Build is necessary, additional CMake scripts is a plus. Documentation: the Boostbook toolchain seems recommended, but not necessary. Is that correct? How free are we? Does the documentation necessarily need to be generated with Boost.Build as well? Test system: something that the regression test suite can eat. What is that format? Is that restricted to Boost.Test run through Boost.Build?

4 3

[BGL] clear_out_edges and MutableIncidenceGraph
by Shaun Jackman 23 Mar '11

23 Mar '11

Hi, adjacency_list implements clear_out_edges, but it's not required by any of the graph concepts: http://www.boost.org/doc/libs/1_46_1/libs/graph/doc/graph_concepts.html Could it be added to MutableIncidenceGraph? Actually, now that I check, I don't see MutableIncidenceGraph on that page either, although it is in graph_concepts.hpp. MutableIncidenceGraph requires remove_edge and remove_out_edge_if. clear_out_edges would fit nicely in here, and clear_in_edges in MutableBidirectionalGraph. Cheers, Shaun

2 1

vault unavailable?
by Oliver Kowalke 23 Mar '11

23 Mar '11

Hello, http://www.boost-consulting.com/vault is unavailable since last night. Will it be available soon? thx, Oliver -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone

1 0

boost at gitorious.org
by Ivan Sorokin 23 Mar '11

23 Mar '11

Hello! http://gitorious.org/boost/svn is not updated since 15 September 2010. I tried to ask its owner Troy Straszheim, but he didn't respond. As I understand this repository is something like an official mirror of boost svn. Does somebody know how to bring it back to live?

4 3

[odeint] gsoc 2011: finalization of odeint
by Mario Mulansky 23 Mar '11

23 Mar '11

Dear boost community, I would like to propose a project for the GSoC 2011, namely to improve the odeint library currently living in the boost sandbox [1] with the (optimistic) goal to put it on a level suitable for a boost review process. I have been working on this library for about one year now, although I couldn't spent much time on it during the last months. The other author of odeint is my colleague Karsten. We have had contact to this mailing list before [2], getting few, but rather positive feedback. The odeint library is a modern, template based, generic, C++ library for integrating (solving) initial value problems of ordinary differential equations. Such an ode is usually given as dx/dt = f(x,t) with x(t=0) = x0. x here is generally vector valued, possibly high dimensional. The easiest method to solve this problem is the Euler-method, where you introduce a small value Dt and then iterate x(t+Dt) = f(x,t)*Dt. Obviously, in real applications one uses more sophisticated methods, a well known class are the Runge-Kutta methods [3], suitable for many cases. Additionally, there exist special methods for system where the time evolution has conserved quantities, called symplectic methods. Furthermore there are methods for stiff systems and many others. For odeint, our first focus lies on Runge-Kutta methods. We have implemented several such methods together with error control routines and lately added dense output functionality. Our very recent code is a generic Runge-Kutta stepper that allows the user to build any Runge-Kutta method at compile time by providing the appropriate constants and is based on fusion::vectors. Karsten wrote a good article on that [4], unfortunately in German. There are also symplectic routines for Hamiltonian systems and we started to add implicit methods as well. We also have unit tests based on the boost test framework and some basic documentation. The library is currently at a level where it can be used in scientific applications and it is indeed in use by us and several other people in an academic environment. The major enhancement of our library over existing ones like gsl or methods from the numerical recipes is that our routines do not depend on the container type used to represent the vector x. So our library works with the usual std::vector<double> and array<double,N> but also with vector< complex<double> > (already a major improvement over gsl and nr, which we get for free, basically) and blitz++ arrays. The major strength of this approach, however, is seen when you want to solve odes on complex networks where you have to implement your own, non-trivial container type representing some kind of network. With odeint you can fully separate the network-implementation from the ode-solving algorithm. Furthermore, we are able to easily employ the power of CUDA devices to solve odes by the help of the thrust library and using thrust::device_vector as our container type. It is also possible to use odeint with arbitrary precision types like the gmp library. Some more, unfortunately slightly out-dated, information about odeint can be found in [5]. What are my plans for a GSoC? 1. There are still some parts of the library which are not satisfactorily designed and need to be rethought. This is where we could really need the help of an experienced mentor to create a clean, well structured library. 2. Add methods with varying order (burlisch stoer). 3. Add dense output and error control to the generic runge-kutta implementation. 4. Add convenience function that allow users an easier access to the libraries capabilities. 5. Create an observer interface that allows the user to analyze properties of the system while integrating the equations. 6. Make everything ready for a boos review. Points 1-5 can surely be realized during the GSoC period, I believe. And whatever time is left I will spend on documentation and review preparation. A few words to myself: I'm currently a PhD student in physics at the University of Potsdam working on problems in the field of nonlinear dynamics. Solving odes is basically every-day-work for me and I have a used many of the routines in odeint for my scientific research. Thanks for reading. I greatly appreciate any feedback! Regards, Mario Mulansky [1] https://svn.boost.org/svn/boost/sandbox/odeint/ [2] http://boost.2283326.n4.nabble.com/library-proposal-odeint-td2670019.html [3] http://en.wikipedia.org/wiki/Runge–Kutta_methods [4] http://www.c-plusplus.de/forum/282091 [5] http://www.codeproject.com/KB/recipes/odeint.aspx

4 6

[context review] Windows ABI requirements
by Holger Grund 23 Mar '11

23 Mar '11

Hi, I've had a very brief look at the x64 ASM Windows code as I think I raised concerns about this before. As always, I'm pretty busy and I probably won't be able to properly review the entire lib. Looking at the fcontext_x86_64_ms_pe_masm.asm it would seem that several ABI requirements are violated and it is not clear to me why this would always work. For instance, the stack is not properly aligned in the call to _exit & set_fcontext. At least _exit can call into user code, IIRC (preterminators & terminator registered via .crt$x* sections) and that code may require a properly aligned stack. I don't see proper unwind descriptors. What happens if an async exception is triggered in the middle of set_fcontext? The x86 Windows version seems to have similar problems. IIRC, the dispatch logic checks for EH registration nodes to be on the stack for x86. What happens if an untrapped exception is trigger in a code path called from the context entrypoint? It would seem that there is an unwind chain via make_fcontext (which doesn't appear to have any unwind description and would therefore assumed to be a leaf function). Is its state necessarily live when set_fcontext is called? What guarantees are there that poking a few fields in the TIB is good enough to properly switch the context? At least FLS has more than one field in the TIB. There's also bookkeeping for whether registered callbacks have been called. Why is saving the register you save enough? Why wouldn't a future version of Windows just take down the process if it detects stack hacks. I believe, the unwinder checks the stack pointer in the unwind context against stack limits and for 8-byte align at each caller in the chain until a barrier is found. How are you certain that kernel32/kernelbase/ntdll don't assume that there's a guard page (where guard page has the Windows defined meaning as in PAGE_GUARD) at the end of the stack? Even if that all works today, how can you be sure it will on Win8, 9 and beyond? To be honest, I don't think a library outside the operating system is not the proper place to do this kind of thing at all (and even if you're part of the OS it's an incredibly hard thing to get right -- UMS had tons of problems in Win7 RTM) BTW, glancing over the docs, an implicit conversion from size_t to protected_stack seems weird, but that might just be a glitch in the doc build. Thanks! -hg

2 4