July 2010 - Boost-users - lists.boost.org

[asio] suitable chunked circular buffer implementation?
by Steve Lorimer 20 Jul '10

20 Jul '10

Hi folks I'm trying (in vain so far) to find a suitable circular_buffer that allows chunked reads/writes which I can use in boost asio. This is my use case: I have 2 threads, Thread 1 reads data from a socket (boost::asio::tp::tcp::socket) and writes it into a buffer (boost::asio::mutable_buffer) Thread 2 reads the buffer and parses out individual messages. Thread 2 shouldn't care about what Thread 1 is doing, specifically whether or not it has completed (or is in the process of) more writes than what Thread 2 has read. All Thread 2 should receive is a small packet of information (the interface being perhaps only iterator begin() and size_t size() ) relating to the chunk of data that was written into the buffer. Reading from the iterator should be safe (ie: it should wrap when it reaches the end of the buffer) Writing into the buffer shouldn't invalidate the iterators that were created previously (which were passed to the 2nd thread) I have an implementation from before I started using boost::asio, which is below. It's quite clearly not standard compliant, but it worked for me. Is there anything out there that works like I've described? Is there an alternative method which I should pursue rather than "chunked circular buffers"? Will I be able to create something like I've pasted below successfully to work with boost asio, or are there issues I'm unaware of? (The reason I ask this question is because I find it surprising that something like this doesn't exist already) Any and all help greatly appreciated. TIA Steve #ifndef _ring_buf_h_ #define _ring_buf_h_ #include <errno.h> #include <boost/thread/mutex.hpp> namespace ring { typedef unsigned char byte; static const uint32_t RING_SIZE = 2 * 1024 * 1024; // 2 meg class iterator { private: const byte* buf; int cur; size_t buf_size; public: iterator() : buf(), cur(), buf_size() {} iterator(const byte* _buf, int _cur, size_t _buf_size) : buf(_buf), cur(_cur), buf_size(_buf_size) {} const byte& operator*() const { return buf[cur % RING_SIZE]; } const byte& operator[](int n) const { return buf[(cur + n) % RING_SIZE]; } iterator& operator+=(int n) { cur += n; buf_size -= n; return *this; } iterator& operator++() { ++cur; --buf_size; return *this; } // post-inc iterator operator++(int) { iterator tmp(*this); ++cur; --buf_size; return tmp; } // pre-inc bool operator==(iterator b) const { return cur == b.cur; } bool operator!=(iterator b) const { return cur != b.cur; } size_t size() const { return buf_size; } int offset() const { return cur; } }; //---------------------------------------------------------------- inline iterator operator+(iterator a, int b) { return a += b; } inline iterator operator+(int b, iterator a) { return a += b; } class buffer { mutable boost::mutex mtx; byte buf[RING_SIZE]; unsigned beg; unsigned end; // may wrap, wrapping is blessed for unsigned types only public: buffer() : beg(), end() { memset(buf, 0, RING_SIZE); } // create an iterator object whose size = unread data in buf iterator data() const { boost::mutex::scoped_lock l(mtx); return iterator(buf, beg % RING_SIZE, end - beg); } // free space in ring unsigned size() const { boost::mutex::scoped_lock l(mtx); return end - beg; } // frees up space in the ring (called once data has been read) void adv(unsigned n) { boost::mutex::scoped_lock l(mtx); beg += n; } // read data from a file descriptor into the buffer // returns: // -errno for error // 0 for EOF // > 0 for successful reads int read(int fd) { mtx.lock(); unsigned free = RING_SIZE - (end - beg); if (!free) { mtx.unlock(); return -ENOMEM; // must never get here } unsigned free_end = std::min(free, RING_SIZE - end % RING_SIZE); unsigned free_beg = free - free_end; iovec io[] = { { (char*)buf + end % RING_SIZE, free_end }, { (char*)buf, free_beg } }; mtx.unlock(); ssize_t n = readv(fd, io, sizeof io / sizeof *io); if (n == -1) return -errno; mtx.lock(); end += n; mtx.unlock(); return n; } }; //---------------------------------------------------------------- } // namespace ring #endif

1 0

[accumulators] density histogram, what is typical way of using it?
by alfC 20 Jul '10

20 Jul '10

Hi, the documentation on accumulator::density is not very clear to me. It seems to fulfil the need for a histogram. Can anyone point at one concrete example using accumulator::density. For example, it seems to determine de bins from the first N samples accumulated. what if you pass less elements than this. Are the bins completely fixed after that? What was the logic behind that design rather than, for example, to somehow declared uniform bins in a range? What if I have to define uniform equispaced bins, do I pass a uniform sequence at the beginning? but then those elements will be counted? How the counters or probabilities are accessed at the end? What else can be done with the density accumulator object, can the cummulative or the inverse pdf be accessed easily within the library. Can it be plugged in into a random number generator as a distribution function. Thank you, Alfredo

2 1

BGL - accessing component subgraphs.
by Adam Spargo 20 Jul '10

20 Jul '10

Any tips on the best way to access a subgraph - ideally without the memory overhead of copying the subgraph to a new object. I want to pass each component of my graph to an algorithm separately, but in some cases the original graph will only have a single or a few component(s), in other cases there will be many. Maybe I will do it differently depending on how many components I have. Any tips welcome. Thanks, Adam. -- Dr Adam Spargo High Performance Assembly Group email: aws(a)sanger.ac.uk Wellcome Trust Sanger Institute Tel: +44 (0)1223 834244 x7728 Hinxton, Cambridge CB10 1SA Fax: +44 (0)1223 494919 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

2 7

students_t distribution with mt19937
by Kraus Philipp 20 Jul '10

20 Jul '10

Hello, I use some distributions with this code: boost::normal_distribution<> l_range(p_mean, p_sd); static boost::mt19937 l_random(static_cast<std::size_t>(time(0))); boost::variate_generator<boost::mt19937&, boost::normal_distribution<> > l_noise( l_random, l_range ); I get my value with l_noise() and it works without any problem. I would like to create now a student_t distribution with boost::math::students_t_distribution<T> l_range(p_v); static boost::mt19937 l_random(static_cast<std::size_t>(time(0))); boost::variate_generator<boost::mt19937&, boost::math::students_t_distribution<> > l_noise( l_random, l_range ); If I run my l_noise() I got a compiler error: error: no match for call to '(boost::variate_generator<boost::random::mersenne_twister<uint32_t, 32, 624, 397, 31, 2567483615u, 11, 7, 2636928640u, 15, 4022730752u, 18, 3346425566u>&, boost::math::students_t_distribution<double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> > >) ()' Can anyone help me please to understand my mistake while using the students_t distribution? I think I need the mt19937 for creating random values and the variate_generator to create a "distribution value" Thanks Phil

2 2

[Random] uniform_int::operator()(Engine &, result_type)
by Hossein Haeri 20 Jul '10

20 Jul '10

Dear all, I was looking at the uniform_int<>'s reference page: http://www.boost.org/doc/libs/1_43_0/doc/html/boost/uniform_int.html and realised that there is no information whatsoever about what the following overload of its operator () does: template<typename Engine> result_type operator()(Engine & eng, result_type n) ; Do I understand it correctly that this generates a number in [min, n] (probably subject to some good conditions)? TIA, --Hossein

2 2

serialization of boost::local_time::tz_database
by Michael Reichlin 20 Jul '10

20 Jul '10

Hi everybody I just started to use boost in some of my applications. I started to serialize my objects. Now I want to serialize my boost::local_time::tz_database object, but the compiler says, there is now serialize function for this object. So do I have to add this function in the header file, or is there any other way to solve this problem. Thanks Michael

1 0

[statechart] : choice_point example no worky
by Noah Roberts 19 Jul '10

19 Jul '10

I've scoured the web for a reason why and have found no links at all, which is surprising since I suspect the documentation is actually in error. The documentation for statechart has this: http://www.boost.org/doc/libs/1_42_ 0/libs/statechart/doc/rationale.html#Limitations (see dynamic choice points) I'm using boost 1.42 so this should be the right document. So I tried it...and it failed with a compiler error that I looked into and makes perfect sense. The state<> template expects the third parameter to be a context, which custom_reaction is apparently not. So I made my own short version which is basically just a copy paste of the example code: #include <boost/statechart/state_machine.hpp> #include <boost/statechart/simple_state.hpp> #include <boost/statechart/state.hpp> #include <boost/statechart/custom_reaction.hpp> #include <boost/statechart/event.hpp> #include <boost/statechart/transition.hpp> #include <boost/mpl/list.hpp> #include <iostream> namespace sc = boost::statechart; struct make_choice : sc::event< make_choice > {}; // universal choice point base class template template< class MostDerived, class Context > struct choice_point : sc::state< MostDerived, Context, sc::custom_reaction< make_choice > > { typedef sc::state< MostDerived, Context, sc::custom_reaction< make_choice > > base_type; typedef typename base_type::my_context my_context; typedef choice_point my_base; choice_point( my_context ctx ) : base_type( ctx ) { this->post_event( boost::intrusive_ptr< make_choice >( new make_choice() ) ); } }; // ... struct MyChoicePoint; struct Machine : sc::state_machine< Machine, MyChoicePoint > {}; struct Dest1 : sc::simple_state< Dest1, Machine > {}; struct Dest2 : sc::simple_state< Dest2, Machine > { Dest2() { std::cout << "Dest2\n"; } }; struct Dest3 : sc::simple_state< Dest3, Machine > {}; struct MyChoicePoint : choice_point< MyChoicePoint, Machine > { MyChoicePoint( my_context ctx ) : my_base( ctx ) {} sc::result react( const make_choice & ) { if ( 0 ) { return transit< Dest1 >(); } else if ( 1 ) { return transit< Dest2 >(); } else { return transit< Dest3 >(); } } }; int main() { Machine machine; machine.initiate(); std::cin.get(); } Resulting error is the same: 1>------ Build started: Project: 2010scratch, Configuration: Debug Win32 ------ 1> scratch.cpp 1>e:\dev_workspace\production.2010rc\3rd_party\boost-1\boost-1.42-msvc10 \include\boost\statechart\simple_state.hpp(893): error C3083: 'context_type': the symbol to the left of a '::' must be a type 1> e:\dev_workspace\production.2010rc\3rd_party\boost-1\boost- 1.42-msvc10\include\boost\statechart\simple_state.hpp(632) : see reference to function template instantiation 'void boost::statechart::simple_state <MostDerived,Context,InnerInitial,historyMode>::deep_construct_inner_imp l_non_empty::deep_construct_inner_impl<InnerList>(const boost::intrusive_ptr<T> &,boost::statechart::state_machine <Machine,InitialState> &)' being compiled 1> with 1> [ 1> MostDerived=MyChoicePoint, 1> Context=Machine, 1> InnerInitial=boost::statechart::custom_reaction <make_choice>, 1> historyMode=has_no_history, 1> InnerList=boost::mpl::list <boost::statechart::custom_reaction<make_choice>>, 1> T=MyChoicePoint, 1> InitialState=MyChoicePoint 1> ] 1> e:\dev_workspace\production.2010rc\3rd_party\boost-1\boost- 1.42-msvc10\include\boost\statechart\state.hpp(81) : see reference to function template instantiation 'void boost::statechart::simple_state <MostDerived,Context,InnerInitial,historyMode>::deep_construct_inner <boost::mpl::list<T0>>(const boost::intrusive_ptr<T> &,boost::statechart::state_machine<Machine,InitialState> &)' being compiled 1> with 1> [ 1> MostDerived=MyChoicePoint, 1> Context=Machine, 1> InnerInitial=boost::statechart::custom_reaction <make_choice>, 1> historyMode=has_no_history, 1> T0=boost::statechart::custom_reaction<make_choice>, 1> T=MyChoicePoint, 1> InitialState=MyChoicePoint 1> ] 1> e:\dev_workspace\production.2010rc\3rd_party\boost-1\boost- 1.42-msvc10\include\boost\statechart\state.hpp(77) : while compiling class template member function 'void boost::statechart::state <MostDerived,Context,InnerInitial>::deep_construct (boost::statechart::state_machine<Machine,InitialState> *const &,boost::statechart::state_machine<Machine,InitialState> &)' 1> with 1> [ 1> MostDerived=MyChoicePoint, 1> Context=Machine, 1> InnerInitial=boost::statechart::custom_reaction <make_choice>, 1> InitialState=MyChoicePoint 1> ] 1> e:\dev_workspace\experimental\2010scratch\2010scratch \scratch.cpp(19) : see reference to class template instantiation 'boost::statechart::state<MostDerived,Context,InnerInitial>' being compiled 1> with 1> [ 1> MostDerived=MyChoicePoint, 1> Context=Machine, 1> InnerInitial=boost::statechart::custom_reaction <make_choice> 1> ] 1> e:\dev_workspace\experimental\2010scratch\2010scratch \scratch.cpp(45) : see reference to class template instantiation 'choice_point<MostDerived,Context>' being compiled 1> with 1> [ 1> MostDerived=MyChoicePoint, 1> Context=Machine 1> ] 1>e:\dev_workspace\production.2010rc\3rd_party\boost-1\boost-1.42-msvc10 \include\boost\statechart\simple_state.hpp(893): error C2039: 'inner_initial_list' : is not a member of 'boost::statechart::custom_reaction<Event>' 1> with 1> [ 1> Event=make_choice 1> ] 1>e:\dev_workspace\production.2010rc\3rd_party\boost-1\boost-1.42-msvc10 \include\boost\statechart\simple_state.hpp(893): error C2039: 'orthogonal_position' : is not a member of 'boost::statechart::custom_reaction<Event>' 1> with 1> [ 1> Event=make_choice 1> ] 1>e:\dev_workspace\production.2010rc\3rd_party\boost-1\boost-1.42-msvc10 \include\boost\statechart\simple_state.hpp(899): error C2039: 'deep_construct' : is not a member of 'boost::statechart::custom_reaction<Event>' 1> with 1> [ 1> Event=make_choice 1> ] 1>e:\dev_workspace\production.2010rc\3rd_party\boost-1\boost-1.42-msvc10 \include\boost\statechart\simple_state.hpp(899): error C2664: 'boost::statechart::simple_state <MostDerived,Context,InnerInitial,historyMode>::deep_construct' : cannot convert parameter 1 from 'const boost::intrusive_ptr<T>' to 'boost::statechart::state_machine<MostDerived,InitialState> *const &' 1> with 1> [ 1> MostDerived=MyChoicePoint, 1> Context=Machine, 1> InnerInitial=boost::statechart::custom_reaction <make_choice>, 1> historyMode=has_no_history 1> ] 1> and 1> [ 1> T=MyChoicePoint 1> ] 1> and 1> [ 1> MostDerived=Machine, 1> InitialState=MyChoicePoint 1> ] 1> Reason: cannot convert from 'const boost::intrusive_ptr<T>' to 'boost::statechart::state_machine<MostDerived,InitialState> *const ' 1> with 1> [ 1> T=MyChoicePoint 1> ] 1> and 1> [ 1> MostDerived=Machine, 1> InitialState=MyChoicePoint 1> ] 1> No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called So, how do I really do this? -- http://crazyeddiecpp.blogspot.com/

2 3

[Asio] io_service destructor never completes
by Ragnar Cederlund 19 Jul '10

19 Jul '10

Hi, I'm having trouble with the io_service destructor never completing. The application I am working on uses one worker thread to handle all actual work to be done including communication on a socket in handler functions posted to the asio queue and another thread to handle the gui of the application. At a certain point in time the program decides that it has nothing more to do and calls the io_service::stop function. Shortly thereafter the thread returns from its call to the io_service::run function. The next thing that happens is that the io_service object goes out of scope and thus the io_service destructor is called. After this the thread signals that it is finished and the application exits. The problem I am seeing is that the destructor never finishes. Or actually, a customer sees this problem, but I have not been able to reproduce this so all the information I have comes from adding (excessive) logging in the boost source. The io_service destructor loops through all services and calls shutdown_service on each one. The win_iocp_io_service handles this call by entering a while loop that should complete when the outstanding_work_ variable becomes zero. Inside the loop, there is a call to GetQueuedCompletionStatus to dequeue a completion packet, which is then tested to see if it contains a valid operation that is then destroyed and the outstanding_work_ variable is decremented. In all the runs I have made, outstanding_work_ is initially 1 when the shutdown_service function is entered. The first and second calls to GetQueuedCompletionStatus come back immediately and the second results in the outstanding_work_ variable being decremented and the loop exits. Everything is fine. At the customer location however, the second call times out instead of completing immediately and it continues to do so for subsequent loop iterations until the application is forcibly closed. There is only one of our customers who gets this problem. The customer runs Windows XP SP3, but I have failed to reproduce this on my own XP test machine. We have tried both Boost version 1.43 and version 1.40 with the same result. In both cases, the customer has the problem but I cannot reproduce it. The asio behavior has changed somewhat between the versions and the description of the problem above is with version 1.43. Does anyone have any idea of what may be going on? Any hints or tips are appreciated! Could we be using asio wrong in some way that could cause this, but we're just lucky most of the time? The customer may be running in a virtual machine, could this affect io completion ports in a way that could be problematic for asio? Cheers, Ragnar

3 5

[Testing] Help with process child
by Romulo 19 Jul '10

19 Jul '10

Hi there, good morning to everyone. I'm using the Boost Test framework to run a set of tests for an application in development and some days ago i faced some problems. In one of my tests, i run what we call "The Processes Test" which in resume is a process launcher. Within the launcher, i run, kill, detach and terminate process and now i am getting lots of ugly and erroneous messages about stack/heap/dead references and unrecognized signals. I wanted to investigate further those errors and on my logs i saw lines like: unknown location(0): fatal error in "testCaseSystemProcess2": child was killed; pid: 20182; uid: 1000; exit value: 9 But indeed i was trying to kill/terminate the children process. Is this ok? If i was really expecting the child process to die, shouldn't boost be quiet about it? Or does it mean the parent process also got killed? Also, on another test case i'm getting: unknown location(0): fatal error in "testCaseSystemProcess1": unrecognized signal Why the signal "id" is not present on the error message? How am i supposed to know which signal was emitted? Thanks for your time and patience, Rômulo.

2 1

[Spirit] Equivalent of limit_d in spirit 2.x
by Roland Bock 19 Jul '10

19 Jul '10

Hi, Spirit 1.8 has a directive called limit_d which allowed to set the value range. Is there an equivalent for this in Spirit 2.x? Thanks and regards, Roland

5 10