- Boost-users - lists.boost.org

Asio and bind. Queuing a bind call.
by Nick.Adamson＠generaldynamics.uk.com 08 Nov '10

08 Nov '10

Hi All. I'm putting together a class which provides a single point of contact to a resource. I'm planning on using the asio IO service in conjunction with a ASIO strand to control flow of method calls to my resource. I've got this working and ASIO has made this so easy. One of the things I've got to do is to be able to pause and queue function calls to the resource if the resource can't service the request. I figured the best way of doing this was putting the result of boost::bind in to some sort of queue when the resource is unavailable and then when it becomes available processing the items in the queue. Is this the best way to do it or does the strand or IO service have a way of doing it? As the result of boost::bind is a template I don't know of an stl or boost container which will store these objects. As there are a number of functions that can be called on my resource with differing arguments I don't think I can use a boost::function as I don't know the arguments to my generic post call. Here's some sudo code which may help to explain it. class CResourceManager { public: // post. // this function takes in a bound function and addes it to the queue of functions to call. // it reserves the sequence of calls by using the io strand. // this means that any function called by this post call is guaranteed to be called in sequence and will be the only function called at that time. template <typename CompletionHandler> void Post(CompletionHandler handler) { if (m_bStrandEnabled) { m_strand.post( handler ); } else { // strand is not enabled. ?????? m_queuedCalls.push( handler ); ????? } } void EnableStrand() { iterate through queue posting functions on to the strand. ?????? code would depend on container used. m_bStrandEnabled = true; } boost::asio::io_service &m_ioService; boost::asio::io_service::strand m_strand; private: bool m_bStrandEnabled; // as a boost bind function is templated we have to work out a way of putting an unknown template in to the queue. queuedfuncitoncontainer m_queuedCalls; } at the moment I don't know what type the "queuedfuncitoncontainer" should be. Because the result of boost::bind is dependent on the function being bound it doesn't seem like I can use stl::queue. Anyone have any ideas how I can do this? I've done a bit of googling and didn't really get any ware. I did look at the boost code to see how strand does it and it seems like its using raw pointers, I didn't really want to go down that road if I can help it. Thanks. Nick. Nick Adamson Software Engineer Advanced Projects and Technology General Dynamics United Kingdom Limited Bryn Brithdir, Oakdale Business Park, Blackwood, South Wales, NP12 4AA Telephone: +44 (0)1495 23 6467 Email: Nick.Adamson(a)generaldynamics.uk.com Website: www.generaldynamics.uk.com <http://www.generaldynamics.uk.com/> To help reduce waste paper, please only print this email if you really need to.

2 2

Re: [Boost-users] Does anybody interested in active object
by Marat Abrarov 08 Nov '10

08 Nov '10

To Dean Michael Berris: Thank you for your patience and detailed answers. > Nope, sorry, s11n_example::connection is not an active object. What it actually is, is an execution > context -- or sometimes people call it a continuation, a generator, or a state machine. Because you're > using the execution context relevant with a particular connection, it doesn't imply that the > operations are performed on a different thread. Ok. May be I'm wrong trying to name it "active object". "Continuation", "state machine", "execution context" - this is very close to what I had in mind. Therefore, I conclude that we understand each other (despite some language barriers with my hand). I thought all of "Continuation/state machine/execution context" was related to the pattern of "active object", hmm... > If you have unlimited number of cores and no limits on the number of threads you can spawn for each > active object you create, then yes. I think the main feature of "Continuation/state machine/execution context" is not saving threads, but organizing flexible and comfortable way of communication between state machines (yes - it is tasks and queues, but wrapped in async method call with callback). That is what I try to implement by means of Boost.Asio. It was chosen "historically" (started from network programming), but for the now times I see it is the best choice :) > Otherwise, you can stick with continuations, asynchronous completion handlers, and just plain data > parallelism. At the "samples" I develop (thanks to Boost.Asio examples - the are the first) the main goals were: 1) share execution context (thread pool) between different "state machines". 2) (make possible to) divide execution contexts of different "state machines" and application layers when it is needed (session and session_manager - not layers but are close to them). 3) develop a simple way of implementing async method call (for the "state machines") with required callback. (with required usage of "custom memory allocation" which can make usage of async methods as cheap as possible) Regards, Marat Abrarov. > Message: 3 > Date: Mon, 8 Nov 2010 10:29:02 +0800 > From: Dean Michael Berris <mikhailberis(a)gmail.com> > To: boost-users(a)lists.boost.org > Subject: Re: [Boost-users] Does anybody interested in active object > Message-ID: > <AANLkTimW0ozMYQ_GfstJw4mV111=2JTc0JSh+0YzLdYF(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Mon, Nov 8, 2010 at 7:11 AM, Marat Abrarov <abrarov(a)mail.ru> wrote: > > To Dean Michael Berris: > >> You'd be better of doing data parallelism with Boost.MPI > > > > No. Because I'm speaking about thread-based parallelism (with shared > > memory access). > > Processed-based parallelism and IPC supplement but don't replace > > thread-based parallelism in general. > > > > Well, which is why I said 'data parallelism'. This implies a whole different kind of programming, > which doesn't typically include threads and synchronization. ;) > > > > >> and/or Boost.Asio's io_service > >> run on multiple threads, and having independent tasks multiplexed > >> across a number of processors (either through processes or threads) > > > > Tie this tasks with the data they share and You will have an active object. > > Nope, sorry, that's not an active object. An Active Object requires that you have a at least one > thread of execution dedicated to the active object, and that you have a synchronous interface to that > object. > > > If the tasks don't have shared data - they don't need an active object. > > Active Objects really have nothing to do with shared data. It's a design pattern that prescribes a way > of isolating the synchronization and serialization of operations applied to an object that is assumed > to be shared across multiple threads of execution. The way the active object does it is by lining up > the operations to be performed asynchronously and perform those in a different thread of execution. > > You can implement an active object without involving shared data -- for example you want to model a > processor, or a math kernel, which does all its work on a single thread because maybe you have a > library underneath that is not thread-safe. In situations like these it makes sense to have an active > object implementation. > > > I think - active object is built on task-based parallelism but it does > > not mean that "active object" should replace "task-based parallelism" > > everywhere. > > > > I think you have it wrong. The active object pattern is a design pattern that doesn't know about task- > based parallelism. Task-based parallelism is a different pattern and different paradigm altogether. > The active object pattern by itself is just a design pattern, nothing more. > > > > >> But what you're describing is not the active object pattern. ;) > > > > http://en.wikipedia.org/wiki/Active_object > > http://en.wikipedia.org/wiki/Active_object_(Symbian_OS) > > What about ma::echo::server::session class - is it an active object? > > > > I haven't looked at your examples, because I was only commenting on the issue regarding massive > parallelism. Maybe when I have enough time I'll look through your examples. ;) > > > > > http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/example/chat/ > > chat_c > > lient.cpp > > Is chat_client an active object? If "yes" then why? > > chat_client::write have no completion callback - and this is not > > enough sometimes (I can say "always", if You don't implementing > > "fire-and-forget" logic which is rather seldom). > > > > No, this is not an active object. An active object has its own lifetime thread. In this situation the > lifetime thread is shared between Boost.Asio's IO subsystem and all the ASIO related handlers. > > Also, you don't need a completion callback for a class to model the Active Object pattern. You can > create methods that return 'void' and just execute the actions asynchronously in its own lifetime > thread. > > > > > http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/example/seria > > lizati > > on/connection.hpp > > s11n_example::connection is an active object and have completion callback... > > but it's much simpler then ma::echo::server::session_manager - I try > > to suggest "samples" that can help to build such a kind of complex > > active objects. > > > > Nope, sorry, s11n_example::connection is not an active object. What it actually is, is an execution > context -- or sometimes people call it a continuation, a generator, or a state machine. Because you're > using the execution context relevant with a particular connection, it doesn't imply that the > operations are performed on a different thread. > > All this is doing is encapsulating callbacks and the protocol implementation into an object. It > exposes a means of asynchronously scheduling operations, but it doesn't hide these details from the > interface. The goal of the active object pattern is to hide the asynchronous nature of the > implementation from the user of the object. > > > > > http://www.drdobbs.com/go-parallel/blog/archives/2009/09/parallelism_s > > ho.htm > > l > >> We are surrounded by parallelism even more so that serial activities. > >> Social and biological systems are immersed in parallelism and these > >> systems have worked out the kinks over millions of years. > >> In biological systems, independent units perform simple computations > >> or activities independently and in parallel. > > and > >> IMPLICATIONS FOR COMPLEX REALTIME SYSTEMS: > >> Complex real-time systems would be a sea of state machine objects > >> interacting with each other. There will be low level state machines > >> which would work together to implement high level state machine behavior. > > > > Here's the place for active object pattern. > > > > If you have unlimited number of cores and no limits on the number of threads you can spawn for each > active object you create, then yes. > Otherwise, you can stick with continuations, asynchronous completion handlers, and just plain data > parallelism. > > The place for an active object is where you have to expose a synchronous interface to a resource which > underneath performs the actions asynchronously. This is like the half-sync half-async pattern which, > once you understand, is what the active object implementation provides. > > -- > Dean Michael Berris > deanberris.com

2 1

Re: [Boost-users] [mpl] Iterator Concept Checking? SOLVED + Recommendation.
by Stirling Westrup 08 Nov '10

08 Nov '10

On Sun, Oct 31, 2010 at 6:16 PM, Stirling Westrup <swestrup(a)gmail.com> wrote: > On Sat, Oct 30, 2010 at 8:57 PM, David Abrahams <dave(a)boostpro.com> wrote: >> >> At Sat, 30 Oct 2010 17:55:35 -0400, >> Stirling Westrup wrote: >>> >>> I am writing some algorithms to operate on MPL sequences, and I would >>> like to use some form of concept checking to ensure that the types >>> passed in that are supposed to be iterators are, in fact, iterators. >>> I have solved the problem I was working on, but as I'm not completely happy with my solution, I thought I'd post it here and make a recommendation for a small change to MPL that would have simplified my work. I discovered while reading about the iterator_category metafunction that the iterator tags had ascending values. ( http://www.boost.org/doc/libs/1_44_0/libs/mpl/doc/refmanual/iterator-catego… ). This is only documented there, and not in the description of the individual iterator categories. In fact, the individual iterator categories are completely misleading as they state that there is a member named 'category' that is convertible to the type of the tag. Since a random iterator is a refinement of a forward iterator and therefor has at least the same set of requirements, it would imply that: BOOST_MPL_ASSERT (( is_convertible<RandomIterator::category,forward_iterator_tag>)) should succeed, and it does NOT. It would seem that one should use iterator_category to determine if something is an iterator or not, but its only actually defined when its type is an iterator, so that doesn't really work. I ended up writing my own version that was defined for all types, like this: struct non_iterator_tag : mpl::int_<INT_MIN> { typedef non_iterator_tag type; }; BOOST_MPL_HAS_XXX_TRAIT_NAMED_DEF(has_category,category,false) template<bool b, typename T> struct category_impl { typedef non_iterator_tag type; static const type::value_type value = type::value; }; template<typename T> struct category_impl<true,T> { typedef typename mpl::iterator_category<T>::type type; static const int value = type::value; }; // just like mpl::iterator_category<T> but can be passed any // type, and returns mplx::not_iterator_tag on bad types. Note // that this value will compare less-than any valid iterator // tag. template<typename T> struct category : category_impl<has_category<T>::value,T> {}; template<typename Iterator> struct is_forward_iterator { typedef typename mpl::greater_equal < typename category<Iterator>::type , mpl::forward_iterator_tag >::type type; }; template<typename Iterator> struct is_bidirectional_iterator { typedef typename mpl::greater_equal < typename category<Iterator>::type , mpl::bidirectional_iterator_tag >::type type; }; template<typename Iterator> struct is_random_access_iterator { typedef typename mpl::greater_equal < typename category<Iterator>::type , mpl::random_access_iterator_tag >::type type; }; The above works, but it relies upon me testing to see if there is an embedded category member, which I suspect is going to get deprecated by iterator_category<> just like all the other members were. It would be far preferable for me if there was an actual non_iterator_tag and iterator_category<> were already defined for all types, so it could both be used in a convenient manner, and so that there was an obvious way to write a specialization in the case that, for instance, one needed the type 'int' to be an iterator. In the very least, please update the documentation to mention the actual relationships between the tags in the discussion about the iterators, and to remove the incorrect 'convertible' description. -- Stirling Westrup Programmer, Entrepreneur. https://www.linkedin.com/e/fpf/77228 http://www.linkedin.com/in/swestrup http://technaut.livejournal.com

3 3

What's so cool about Boost.MPI?
by David Abrahams 08 Nov '10

08 Nov '10

Just wanted to call some attention to this article I wrote about Boost.MPI: http://daveabrahams.com/2010/09/03/whats-so-cool-about-boost-mpi/ I think it might be useful to link that from the Boost.MPI docs, too. -- Dave Abrahams BoostPro Computing http://www.boostpro.com

12 12

Re: [Boost-users] Does anybody interested in active object
by Marat Abrarov 08 Nov '10

08 Nov '10

To Dean Michael Berris: > You'd be better of doing data parallelism with Boost.MPI No. Because I'm speaking about thread-based parallelism (with shared memory access). Processed-based parallelism and IPC supplement but don't replace thread-based parallelism in general. > and/or Boost.Asio's io_service > run on multiple threads, and having independent tasks multiplexed > across a number of processors (either through processes or threads) Tie this tasks with the data they share and You will have an active object. If the tasks don't have shared data - they don't need an active object. I think - active object is built on task-based parallelism but it does not mean that "active object" should replace "task-based parallelism" everywhere. > But what you're describing is not the active object pattern. ;) http://en.wikipedia.org/wiki/Active_object http://en.wikipedia.org/wiki/Active_object_(Symbian_OS) What about ma::echo::server::session class - is it an active object? http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/example/chat/chat_c lient.cpp Is chat_client an active object? If "yes" then why? chat_client::write have no completion callback - and this is not enough sometimes (I can say "always", if You don't implementing "fire-and-forget" logic which is rather seldom). http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/example/serializati on/connection.hpp s11n_example::connection is an active object and have completion callback... but it's much simpler then ma::echo::server::session_manager - I try to suggest "samples" that can help to build such a kind of complex active objects. http://www.drdobbs.com/go-parallel/blog/archives/2009/09/parallelism_sho.htm l > We are surrounded by parallelism even more so that serial activities. > Social and biological systems are immersed in parallelism and these > systems have worked out the kinks over millions of years. > In biological systems, independent units perform simple computations > or activities independently and in parallel. and > IMPLICATIONS FOR COMPLEX REALTIME SYSTEMS: > Complex real-time systems would be a sea of state machine objects > interacting with each other. There will be low level state machines > which would work together to implement high level state machine behavior. Here's the place for active object pattern. Thanks for that attention that you have given to my question. Marat Abrarov. > -----Original Message----- > Date: Sun, 7 Nov 2010 12:58:06 +0800 > From: Dean Michael Berris <mikhailberis(a)gmail.com> > To: boost-users(a)lists.boost.org > Subject: Re: [Boost-users] Does anybody interested in active object > pattern? > Message-ID: > <AANLkTi=PTkqO2ngdYcNNbp12hNnS=8=PxK3YsccajpFV(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Sat, Nov 6, 2010 at 10:56 PM, Marat Abrarov <abrarov(a)mail.ru> wrote: > > > > Does anybody interested in active object pattern for massive parallelism > by means of C++? (Intel TBB doesn't offer a > > similar approach) > > > > I'm not sure how you think active objects help for massive parallelism > because the active object pattern implies that you're serializing > operations on a single object. > > I've used Boost.Asio to implement Active Objects but only in cases > where it makes sense -- like for example a concurrent accumulator or a > concurrent resource handle (log sink for example). You'd be better of > doing data parallelism with Boost.MPI and/or Boost.Asio's io_service > run on multiple threads, and having independent tasks multiplexed > across a number of processors (either through processes or threads). > > I remember reading about Microsoft's efforts (through Herb Sutter) to > introduce an "active" extension to Microsoft Visual C++, where you can > easily make classes that instantiate to active objects. I'm not sure > where that effort is now though. > > HTH > > -- > Dean Michael Berris > deanberris.com >

2 1

[GIL] Adding base_channel_type metafunction
by Nathan Crookston 08 Nov '10

08 Nov '10

All, Several months ago I submitted a patch to prevent warnings when performing certain channel conversions [1]. After some discussion with Christian a part of the patch was removed in favor of using the unsigned_integral_max_value metafunction. After tweaking my specializations of that metafunction my code compiled fine. Recently I modified my code to use GIL more extensively. I'm pleased with the reduced code footprint (a 5000 line file was reduced to ~150 lines), but unfortunately there are more warnings due to unsafe conversions with my now-more-extensive testing. Attached is a file which adds a base_channel_type metafunction which correctly determines the type which the channel constructor is expecting. I believe this is more correct than using unsigned_integral_max_value as the value_type of *max_value is not the same as the channel type for, e.g., 8-bit unsigned channels. Something like this metafunction has also been requested recently [2]. The following demonstrates its use for the built-in channel type models I'm aware of: ///////////BEGIN CODE//////////////////// #include <boost/cstdint.hpp> #include <boost/gil/gil_all.hpp> #include <boost/static_assert.hpp> #include <boost/type_traits/is_same.hpp> namespace gil = boost::gil; typedef gil::packed_image3_type<boost::uint16_t, 7,7,2, gil::bgr_layout_t>::type bgr772_image_t; typedef gil::kth_element_type<bgr772_image_t::value_type,0>::type b7_element_type; typedef gil::base_channel_type<b7_element_type>::type b7_type; BOOST_STATIC_ASSERT((boost::is_same<b7_type, boost::uint8_t>::value)); typedef gil::base_channel_type<gil::channel_type<gil::gray32f_pixel_t>::type>::type gray32f_type; BOOST_STATIC_ASSERT((boost::is_same<gray32f_type, float>::value)); typedef gil::base_channel_type<gil::channel_type<gil::rgb8_pixel_t>::type>::type rgb8_type; BOOST_STATIC_ASSERT((boost::is_same<rgb8_type, boost::uint8_t>::value)); int main() {} /////////END CODE///////////////// Adding the metafunction and using it to cast values prior to channel construction has removed the warnings from my testing. The diff file also contains changes to channel_algorithm.hpp to suppress warnings using the supplied metafunction. I'd be happy to write additional test cases, trac tickets, etc. The changes have been tested with g++3.4, g++4.3 and VC8 (VS2005). Thanks, Nate [1] <http://lists.boost.org/boost-users/2010/06/60066.php> [2] <http://lists.boost.org/Archives/boost/2010/07/169159.php>

2 9

the name of installed Boost libraries, with or without '-gcc'
by weining hao 07 Nov '10

07 Nov '10

Hi, I have installed boost 1.33-1 in SLES (SUSE Linux Enterprise Server) 11 manually by bjam 3.12, and in SLES10 sp2 via YAST2 (with the defalut bjam 3.1.4). I found that the installed libraries in SLES 11 are always with the postfix '-gcc', which is not appeared in the libraries in SLES 10 sp2 (For example, "libboost_filesystem-gcc.o" in SLES 11, but "libboost_filesystem.o" in SLES10 sp2). The '-gcc' also exists in the library name in SLES 9.3. Is it configurable to be with or without '-gcc', and in which file? Is it related to the SLES version, or bjam version? It is appreciated that you give some hints! Best regards, Weining

2 1

Does anybody interested in active object pattern?
by Marat Abrarov 07 Nov '10

07 Nov '10

Hello, Boost C++ Libraries' community. I'm trying to develop the idea of (most effective) building active objects by means of Boost C++ Libraries (not as libpoet does). I chose for this Boost.Asio as the only library supports flexible "custom memory allocation" mechanism. Here is my project: https://sourceforge.net/projects/asio-samples/ And here is short description of the concept: https://think-async.com/Asio/Examples This project isn't a library and won't be it. It is something like advanced examples for Boost.Asio. And I hope this "advanced examples" are very close to production-level code. Does anybody interested in active object pattern for massive parallelism by means of C++? (Intel TBB doesn't offer a similar approach) Regards, Marat Abrarov.

3 5

[iostreams] compressing and decompressing in memory
by Brian Budge 07 Nov '10

07 Nov '10

I've been searching around for examples for how to compress and decompress using the zip or gzip iostreams compressors into/out of memory buffers. Has anyone tried this? Would you use basic_array for this? Thanks, Brian

2 4

sizeof(boost::fusion:map)
by Robert Lupton the Good 06 Nov '10

06 Nov '10

I was very excited to find the boost::fusion library, expecting that I could use it to implement structs with introspection. I still think I can, but I was disappointed to see that I'm paying a size penalty. I'd have expected all the book-keeping to be handled at compile time. The following code prints sizeof(Foo) = 12 sizeof(FooS) = 4 with g++ 4.2 or 4.4 on an os/x 10.6 or Centos box. Here Foo is boost::fusion::map<boost::fusion::pair<Name, int> > Where are the extra 8 bytes going? With more complex boost::fusion objects I see an overhead of up to 16 bytes. gdb indicates that foo.data.vec has the desired size, but foo.data is larger, and foo is larger still; however, it doesn't indicate any data members --- is this some sort of alignment issue? Is there a way to generate a "packed boost::fusion::map" without this space overhead? R #include <iostream> #include "boost/fusion/container.hpp" struct Name {}; int main() { typedef boost::fusion::map<boost::fusion::pair<Name, int> > Foo; struct FooS { int age; }; std::cout << "sizeof(Foo) = " << sizeof(Foo) << " sizeof(FooS) = " << sizeof(FooS) << std::endl; }

5 11