October 2009 - Boost - lists.boost.org

[task] reschedule_until
by Baxter Jeremy 05 Oct '09

05 Oct '09

> Date: Fri, 2 Oct 2009 18:59:10 +0200 > From: k-oli(a)gmx.de > Subject: Re: [boost] [task] > > Second - I'm finishing boost.task-0.4.0 which replaces > this_task::reschedule_until() by this_task::block(). > block() suspends the current task without blocking the > worker-thread. The > blocked task gets rescheduled by the internal scheduler > (returning from > block()): > > while ( ! condition) > this_task::block(); > > Version 0.4.0 allows to synchronize or exchange messages > between the tasks > without deadlocking the threadpool (that means amount of > tasks >> worker- > threads in pool). > > regards, > Oliver Thanks - I'll steer clear of using reschedule_until at the moment. Being able to block a task without locking out the worker thread would be useful for me, but I suspect my requirement may be rather different to yours. I'm pretty much assuming amount of tasks == worker-threads in pool. My main interest in a thread pool is to avoid creating and destroying a new thread each time a task has to run but I suspect I may use task or threadpool it this way initially and see what the overhead is like. Jeremy The information contained in this E-Mail and any subsequent correspondence is private and is intended solely for the intended recipient(s). The information in this communication may be confidential and/or legally privileged. Nothing in this e-mail is intended to conclude a contract on behalf of QinetiQ or make QinetiQ subject to any other legally binding commitments, unless the e-mail contains an express statement to the contrary or incorporates a formal Purchase Order. For those other than the recipient any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance on such information is prohibited and may be unlawful. Emails and other electronic communication with QinetiQ may be monitored and recorded for business purposes including security, audit and archival purposes. Any response to this email indicates consent to this. Telephone calls to QinetiQ may be monitored or recorded for quality control, security and other business purposes. QinetiQ Limited Registered in England & Wales: Company Number:3796233 Registered office: 85 Buckingham Gate, London SW1E 6PD, United Kingdom Trading address: Cody Technology Park, Cody Building, Ively Road, Farnborough, Hampshire, GU14 0LX, United Kingdom http://www.qinetiq.com/home/notices/legal.html

2 1

[task] reschedule_until
by Baxter Jeremy 05 Oct '09

05 Oct '09

- my example and some of Vicente's comment snipped - > > I think that it will not to much complex to provide a > function that submit a task that must run on the context of a > different thread. Would this kind of submit function help you? > I'm not sure that it would help, I think that I could construct a more complex example with more tasks and threads that would still deadlock. In general I think it ought to be a requirement that the threadpool works if it has only a single thread (in which case the above would not work). > > I'm not sure that there is a general solution to this > problem without > > using more threads so that any potentially runnable task will get a > > chance. > > You are right this should be a deadlock problem with the > current implementation. > I don't think however that having more threads would solve > the issue, but just reduce its frequency, as the issue > appears when tasks T1 and T2 are handled on the same thread. > I agree, that is why I was not keen on reschedule-until. If blocking is used I think an unexpected deadlock could occur unless all blocked tasks can be resumed in any order, not strictly in reverse order of their reschedule calls as in the current implementation. > In order to avoid the issue we need to avoid two blocking > tasks to be executed on the same thread. The user should state > blocking task on the submit function. The scheduler will not > take a blocking task while in the reschedule_until function. > Of course, in order to be able to avoid any deadlock > associated to the fact we are using a thread pool, the number > of threads must grow, if there is no free worker threads able > to handle the new blocking task. The reschedule_until > function should throw an exception when the task is not a > blocking task. > > This should solve the issue, but introduce a burden at the > user level, i.e. specify if the task can block or not. > > Jeremy, Oliver, what do you think? > > Best, > Vicente > > I think Oliver's intention to use a fiber style scheduler is the right one. To avoid problems all possible mechanisms of blocking would need to be considered and protected against and the temptation would be to declare most tasks as blocking. Even if tasks don't block they may run on the thread for a long time before returning. A dynamic thread pool would also be a solution, starting more threads as tasks block and cleaning them up when the queue is empty. This is the approach used in the Java Executors class (http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/Executors. html). I'm not sure if there is a way to detect the difference between threads being busy and being blocked as in general you would like to keep the number of unblocked threads down to the number of cores. Jeremy Jeremy Baxter Lead Researcher, UAVs & Autonomy Tel: 01684 894801 email: jwbaxter(a)QinetiQ.com www.QinetiQ.com QinetiQ - Delivering customer focused solutions Please consider the environment before printing this email. The information contained in this E-Mail and any subsequent correspondence is private and is intended solely for the intended recipient(s). The information in this communication may be confidential and/or legally privileged. Nothing in this e-mail is intended to conclude a contract on behalf of QinetiQ or make QinetiQ subject to any other legally binding commitments, unless the e-mail contains an express statement to the contrary or incorporates a formal Purchase Order. For those other than the recipient any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance on such information is prohibited and may be unlawful. Emails and other electronic communication with QinetiQ may be monitored and recorded for business purposes including security, audit and archival purposes. Any response to this email indicates consent to this. Telephone calls to QinetiQ may be monitored or recorded for quality control, security and other business purposes. QinetiQ Limited Registered in England & Wales: Company Number:3796233 Registered office: 85 Buckingham Gate, London SW1E 6PD, United Kingdom Trading address: Cody Technology Park, Cody Building, Ively Road, Farnborough, Hampshire, GU14 0LX, United Kingdom http://www.qinetiq.com/home/notices/legal.html

2 1

Re: [boost] [rdb] rdb/persistent integration
by strasser＠uni-bremen.de 05 Oct '09

05 Oct '09

>> also, you could use more than one resource managers at a time, so you could >> have distributed transactions between two databases, between a file and a >> database, etc., and can have references across resources. (a locator can >> reference an object in any resource, and can be stored as part of an object) >> > Very interesting project. At this point my feeling is that there is room > to slip not one but two of my libs under yours ! > what`s the other library? btw, I wasn`t suggesting we should merge our libraries. a stand-alone sql db layer definitely makes sense. even a standalone o-r mapper might make sense, depending on what kind of user interface you planned for it. it might e.g. be better for some use cases to give the user more control over when and how actual SQL is produced and objects are stored than my library does. much like my resource manager uses Boost.Serialization, but doesn`t replace Boost.Serialization. remember that my intent was a user interface as close as possible to using regular objects in memory, so there is no "store this object now" function. the resource manager is free to cache and postpone accesses to the storage engine, as long as it guarantees that the object is stored on transaction::commit the latest. but we should work together on "linking" the libraries, i.e. implementing a boost.persistent resource manager that uses your o-r mapper, in the current design. or merge your o-r mapper library with my library, but leave the sql db layer a seperate library. depending on the o-r mapper interface you planned.

1 0

Looking for a name for a Subjet-Entity-Role library
by vicente.botet 05 Oct '09

05 Oct '09

Hi, I'm developing a library based on the Object Role pattern. The library use the following concepts: Entity, Role, Subject, Cloture. Its main goal is to help on subjetive programming. I was started to name this library SER (for Subject-Entity-Role). Next follows some Entity-Role-Subject declarations class Person : public entity<Person> {...}; template <typename Base> class PersonRoleTmpl : public basic_role<Person, Base> {...}; class PersonRole : public PersonRoleTmpl<> {...};class Employee : public role<Employee,PersonRole> {...}; class Salesman : public Employee {...}; class Developer : public Employee {...}; class Manager : public Employee {...}; class Guarantor : public role<Customer, PersonRole> {...}; class Customer : public role<Customer, PersonRoleTmpl<entity<Customer> > > {...}; class CustomerRole : public basic_role<Customer> {...}; class Borrower : public role<Borrower, CustomerRole> {...}; class Investor : public role<Investor, CustomerRole> {...}; class CustomerAndEmployee : public subject<Person, roles<Customer,Employee> > { ... }; Any sugestions for the name of the library, 'Subjetive', 'SER'? Best, _____________________ Vicente Juan Botet Escribá

5 7

[sandbox] Mysterious (?) commit messages, 'm', 'a', 'd'
by Niels Dekker - address until 2010-10-10 04 Oct '09

04 Oct '09

There appear commit messages of a single letter: 'd', 'm', 'M', 'a'. Or combined: "m/a". It seems like they only appear in the sandbox. https://svn.boost.org/trac/boost/timeline What do they mean? (Maybe "e_r" can tell us.) FWIW, I'd rather see more "human friendly" commit messages. Thanks in advance, Niels -- Niels Dekker http://www.xs4all.nl/~nd/dekkerware Scientific programmer at LKEB, Leiden University Medical Center

2 1

GIL extensions home and guide
by Mateusz Loskot 04 Oct '09

04 Oct '09

Hi, May I ask for update where is most current home for Boost/Adobe GIL library extensions [0]? What's the difference beetween extensions in Boost SVN [1] and those hosted in Google Code [2] ? Second, what document should one refer to learn about developing new I/O extensions for GIL? The GIL Design Guide [3] is quite dated, from 2007 [0] http://opensource.adobe.com/wiki/display/gil/Contribute+to+GIL [1] https://svn.boost.org/trac/boost/browser/trunk/boost/gil/extension/ [2] http://code.google.com/p/gil-contributions/ [3] http://www.boost.org/doc/libs/1_40_0/libs/gil/doc/html/gildesignguide.html Best regards, -- Mateusz Loskot, http://mateusz.loskot.net Charter Member of OSGeo, http://osgeo.org

2 2

[rdb] 0.0.09
by Jean-Louis Leroy 04 Oct '09

04 Oct '09

In this version : * EXISTS * consistency check in INSERT...SELECT * DISTINCT, ALL * transactions The transaction stuff is pellicular. It simply maps to vendor calls. I am hesitating a bit, either make a common interface for the most common operations (like execute) directly in each backend (aside with vendor-specifc calls) or put them in a separate wrapper or even decide that it's not in the scope. There's more redesign in the select from which I think I can get an elegant implementation of vendor-specific features (like LIMIT). The idea is to pass all the data in a fusion map, allowing vendor specific parts of the statement to slip their own data inside it. In fact this also is how DISTINCT works. I dropped the composition of lists via chained calls to operator (). It's all preprocessor-based now. The () chain system was limited by FUSION_MAX_VECTOR_SIZE anyway. I have installed MySQL on my computer. I am still accessing it via ODBC but I'll probably start the native support soon. Yesterday I tried to upload and just when I pushed the upload button the vault server crashed. I suspect that it was my upload that triggered it. J-L

6 15

[trac] Make me property_tree owner
by Sebastian Redl 04 Oct '09

04 Oct '09

Hi, Could somebody who has the authority to do so please make me (cornedbee) the owner of property_tree? Currently, all PTree tickets are still assigned to kaalus, who is the original author. Sebastian

2 1

[Fusion] port to c++0x
by Christopher Schmidt 04 Oct '09

04 Oct '09

I participated in Google's 'Summer of Code' for Boost this year. My project was to port Boost.Fusion to c++0x. The program is over now and I would like to give you a short report on my work and put my code up for comments. In a nutshell, there is rvalue support, native variadic templates are used if possible and the public classes and (meta-) functions are guarded by a set of static asserts that detect most invalid (template) arguments. I also added certain new functionality to provide an easier, faster and more intuitive programming interface. My implementation is pretty much backwards compatible and all test cases pass on gcc 3.4/4.4 and vc 9. Here is a short overview of the most striking changes besides the implementation of raw c++0x support. I removed the deque container and the extension-classes. The deque container is undocumented and, in my opinion, does not go well with the other container due to its intrinsic extension mechanism. There also should not be a performance gain compared to a regular fusion::joint_view in combination with two fusion::vector. The extension classes are pretty much Boost.Proto specific. At the moment they seem to be out of place, especially as they are undocumented, unmaintained and not even part of the regular file structure. I would like to see them either fully documented (with the underlying concept of segmented sequences) or in the detail namespace of Boost.Proto. I added support for associative iterators. An associative forward sequence shall provide an iterator type which implements the extension backend of the intrinsic (meta-)functions fusion::result_of::value_of_data, fusion::deref_data and fusion::result_of::key_of. This has the advantage that all view classes, even fusion::iterator_range, are able to adapt the underlying sequence's associative-ness. fusion::transform_view, as a special case, got an additional template parameter. This parameter specifies whether the view should model either the forward concept only or the associative and the forward concept. This design breaks consistency with Boost.MPL. Unfortunately these changes are crucial and I would rather risk this break with Boost.MPL than missing that important feature. Using the old implementation, there is no way to avoid explicitly converting from a view back to an associative sequence to preserve the associative-ness. This leads to a nasty runtime overhead due to evaluation of every element of the view and copy-/move-constructing it. Besides, other changes include: -If the compiler supports rvalue references, all fusion container have a template constructor (taking either const lvalue or 'templated' rvalue references). -If the compiler supports variadic templates, fusion::cons, fusion::pair and fusion::single_view have a variadic template constructor (taking either const lvalue or 'templated' rvalue references) that allows emplace construction. -The view classes adapt the full category of their underlying sequences. As mentioned above, the associative-ness is preserved, but also, if possible, the random-access or bidirectional trait. -The result_of::-metafunctions accept r- and lvalue references arguments. If an argument is not reference-qualified, it will be considered to be an lvalue reference. All existing extension implementations must be changed to adapt this new behaviour. -The algorithm functions now also take (and correctly forward) non-const reference arguments. -Fusion iterators (with reference compatible sequence types and a very same index) and views (with reference compatible sequence types) have a working operator= that correctly reassigns the underlying sequence references. -Fusion now also adapts MPL iterators. If, and only if, the <boost/fusion/adapted/mpl.hpp>-header is included, mpl sequences and iterators are recognized as fusion sequences and iterators. -There are new headers to adapt std::(tr1::)array or std::(tr1::)tuple. The overall compile-time performance of my code is satisfying. Compiling a compilation unit that includes all possible fusion headers takes on gcc 4.4 slightly more time than the official fusion implementation. If native variadic templates are present, the compilation is a lot faster (usually up to 3 to 6 times), regardless of the value of FUSION_MAX_xxx_SIZE. The compile-time performance of actual fusion code is mixed. The intrinsic container and iterator (meta-)functions compile a little bit slower compared to the official fusion implementation. This is due to explicit removal of the references on the arguments of all result_of::metafunctions and some additional abstraction layer. If the compiler support rvalue references, there has to be a rather ugly and slow type evaluation/transformation to avoid issues with rvalue references on non-array, non-class types (see /support/internal/ref.hpp and 8.5.3p5 of the current c++0x draft for more details). On the other hand, I added several optimizations (such as result preevaluation), and, if variadic templates are present, the actual instantiation of fusion containers is a lot faster. All in all, the compile-time performance is a little bit worse if c++0x features are not present. If c++0x features are present, the code should compile a little bit faster in most cases. For example, the lambda example of the Boost.Proto documentation, which internally heavily relies on Boost.Fusion, takes on my machine 2.6 seconds to compile with the 'old' fusion on gcc 4.4, 3 seconds with my port without -std=c++0x, and 2.1 seconds with c++0x features being present. The overall performance might be even better once Boost.MPL adapts native variadic templates and the compiler supports template aliases. There have been tons of other changes, mostly bugfixes, optimizations (in terms of compile time of course) and removals of redundancy. Whenever I had the choice, I tried to prefer clean, redundancy-free instead of heavily optimized code. The overall performance loss is not noticeable though, and most compile-time optimizations that go beyond the as-little-template-instantiations-as-possible paradigm are pretty much compiler dependent anyway. There is still some work to do. I did not add c++0x-specific test cases and there are several parts of the documentation which need to be updated. By the way, are there any official guidelines for documenting c++03/c++0x-specific differences? For now, I would like to hold off on implementing these changes until Boost.TypeTraits adapts c++0x. Right now I implemented missing or incorrectly behaving trait functions in the fusion::detail-namespace. I would rather like to use the official ones, at least in the documentation and in the test cases. Besides that, I think my implementation is trunk-ready. If there are no objections, I would like to finish the outstanding work as soon as possible, and then merge my code with the trunk. Of course I will provide long-term support for the project. In fact, I would love to become an official fusion maintainer. You can find the code in the boost sandbox (/sandbox/SOC/2009/fusion). If you test the code with a compiler that has decltype support, please use the HEAD of the trunk rather than the official 1.40 distribution package to compile your code. Best regards Christopher

6 17

Review Request: Boost.Itl; The Interval Template Library
by Joachim Faulhaber 03 Oct '09

03 Oct '09

Dear developers, the Interval Template Library is now in a state where I think it does conform the requirements and guidelines for boost libraries fairly well. To decide, whether it's design, implementation and overall usefulness is sufficient to be accepted in the boost libraries collection is now up to you. In the last round of refactoring I concentrated on efficiency and optimization issues. The implementation of the itl's interval containers is still based on std::set and std::map, but algorithms are now exploiting hinted insert and are manipulating intervals and values in place instead of copying in many instances. This way, it has been possible to improve performance significantly. Along the way of implementing optimizations I have documented the complexity guarantees for all important functions of the library's interface. Please note that the original ITL as presented in May 2008 has been split into 3 parts and only the *first* part is submitted for review: itl : Core library, interval containers, SUBMITTED FOR REVIEW Additional parts are: itl_xt : Extensions like histories and generalized crosstables validate: The law based test automaton (LaBatea). A validation tool using laws aka axioms. *itl* is a complete library containing all required ingredients particularly a separate test suite (NOT using LaBatea) and the quickbook documentation. The code is successfully compiled and tested with gcc-3.4.4, 4.1.0, 4.3.2 and msvc-8.0, 9.0. I have uploaded the review version as itl_3_1_0.zip to the vault in section containers. http://www.boostpro.com/vault/index.php?action=downloadfile&filename=itl_3_… Another file itl_plus_3_1_0.zip contains all three parts (itl, itl_xt and validate). http://www.boostpro.com/vault/index.php?action=downloadfile&filename=itl_pl… Reviewers interested in the law based testing part may have to refer to this file to examine those tests. The law base testing examples compile and run with gcc-4.3.2, msvc-8.0 and 9.0. LaBatea does not compile with gcc-4.1.0 or older. The quickbook generated documentation is included in the zip files and is also available online at http://herold-faulhaber.de/boost_itl/doc/libs/itl/doc/html/index.html Sources are also available form the sandbox: https://svn.boost.org/svn/boost/sandbox/itl/ To close my request for review, let me take the opportunity to do some advertisement ;-) focusing on some of the itl's highlights. * The Itl emerged out of real world software development extracting generic functionality from numerous use cases of the date time problem domain. (http://www.herold-faulhaber.de/boost_itl/doc/libs/itl/doc/html/boost_itl/ex…) But the need for the same data structures occurs in other fields as well. E.g. Zachary Turner uses a compressed_bitmap similar to an itl::interval_set to manage very large file system allocation bitmaps and gives more use cases here (http://lists.boost.org/Archives/boost/2009/06/152817.php) * Aggregation on overlap not only provides a useful feature to compute aggregation results on interval_maps like statistics, it also leads to a generalized addition operation on maps that exhibits interesting semantical properties. E.g. (http://herold-faulhaber.de/boost_itl/doc/libs/itl/doc/html/boost_itl/semant…) * The itl's design is based on sets of laws (aka axioms) that are described in the documentation. (http://herold-faulhaber.de/boost_itl/doc/libs/itl/doc/html/boost_itl/semant…) These law sets not only are giving a formal specification of the library. Using the law based testing tool (LaBatea) (https://svn.boost.org/svn/boost/sandbox/itl/boost/validate) allows to validate the library code for every specified law. * For the itl, 18 laws have been validated on the 7 container templates with varying instantiation types for template parameters, running a total of 719 different law instances (each of them for large numbers of randomly generated test data). (https://svn.boost.org/svn/boost/sandbox/itl/libs/validate/example/) This gives everyone a method to check semantical properties of the library in a way that is more powerful than traditional unit tests. To get the 'complete picture' of the library's interface you may want to refer to the synoptic function table first (http://www.herold-faulhaber.de/boost_itl/doc/libs/itl/doc/html/boost_itl/in…) before looking at more specific topics. Hoping to have quickened your appetite to take a closer look at this library before and during the review process. best regards Joachim Faulhaber

6 28