- Boost-users - lists.boost.org

Re: [Boost-users] large data sets and BOOST_FOREACH / boost:bind don'tmix ?
by pete＠pcbartlett.com 05 Feb '10

05 Feb '10

I would hazard a guess that you are using MSVC with _SECURE_SCL left as the default ON. Here are my results with _SECURE_SCL on: Time: BOOST_FOREACH(Fred& fred, vec) { fred.inc(); } 1.406 Time: std::for_each(vec.begin(),vec.end(),boost::bind(&Fred::inc,_1) ); 0.515 Time: for (std::vector::iterator i = vec.begin(); i < theEnd ; i++) { (*i).inc(); } 1.313 Time: std::for_each(vec.begin();vec.end(),std::mem_fun_ref(&Fred::inc)) 0.516 Time: for (size_t i = 0; i < theSize ; i++) { vec[i].inc(); } 0.391 And with _SECURE_SCL off: Time: BOOST_FOREACH(Fred& fred, vec) { fred.inc(); } 0.36 Time: std::for_each(vec.begin(),vec.end(),boost::bind(&Fred::inc,_1) ); 0.515 Time: for (std::vector::iterator i = vec.begin(); i < theEnd ; i++) { (*i).inc(); } 0.344 Time: std::for_each(vec.begin();vec.end(),std::mem_fun_ref(&Fred::inc)) 0.516 Time: for (size_t i = 0; i < theSize ; i++) { vec[i].inc(); } 0.375 BODY { font-family:Arial, Helvetica, sans-serif;font-size:12px; }So you can see that going through a function pointer has an abstraction penalty - but this is basically the same whether it be through bind or std::mem_fun_ref (haven't checked but assume they compile to same optimized code) Iterators are faster than indexes if _SECURE_SCL is off and slower if it is on - no new news here. ForEach has a small penalty over iterators, but is rare that this matters. Pete On Fri 05/02/10 15:13 , Avi Bahra wrote: In my application I have very large data sets, I was using BOOST_FOREACH and boost bind. But I was surprised that performance wise they were the worst ? Take the following test: class Fred { public: Fred(int i = 0) : i_(i) {} void inc() { i_++;} private: int i_; }; BOOST_AUTO_TEST_CASE( test_loop ) { size_t vecSize = 200000000; std::vector vec; vec.reserve(vecSize); for (size_t i = 0; i < vecSize ; i++) { vec.push_back(Fred(i));} boost::timer timer; BOOST_FOREACH(Fred& fred, vec) { fred.inc(); } cout

1 0

large data sets and BOOST_FOREACH / boost:bind don't mix ?
by Avi Bahra 05 Feb '10

05 Feb '10

In my application I have very large data sets, I was using BOOST_FOREACH and boost bind. But I was surprised that performance wise they were the worst ? Take the following test: class Fred { public: Fred(int i = 0) : i_(i) {} void inc() { i_++;} private: int i_; }; BOOST_AUTO_TEST_CASE( test_loop ) { size_t vecSize = 200000000; std::vector<Fred> vec; vec.reserve(vecSize); for (size_t i = 0; i < vecSize ; i++) { vec.push_back(Fred(i));} boost::timer timer; BOOST_FOREACH(Fred& fred, vec) { fred.inc(); } cout << "Time: BOOST_FOREACH(Fred& fred, vec) { fred.inc(); } " << timer.elapsed() << "\n"; timer.restart(); std::for_each(vec.begin(),vec.end(),boost::bind(&Fred::inc,_1) ); cout << "Time: std::for_each(vec.begin(),vec.end(),boost::bind(&Fred::inc,_1) ); " << timer.elapsed() << "\n"; timer.restart(); std::vector<Fred>::iterator theEnd = vec.end(); for (std::vector<Fred>::iterator i = vec.begin(); i < theEnd ; i++) { (*i).inc(); } cout << "Time: for (std::vector<Fred>::iterator i = vec.begin(); i < theEnd ; i++) { (*i).inc(); } " << timer.elapsed() << "\n"; timer.restart(); std::for_each(vec.begin(),vec.end(),std::mem_fun_ref(&Fred::inc) ); cout << "Time: std::for_each(vec.begin();vec.end(),std::mem_fun_ref(&Fred::inc)) " << timer.elapsed() << "\n"; timer.restart(); size_t theSize = vec.size(); for (size_t i = 0; i < theSize ; i++) { vec[i].inc(); } cout << "Time: for (size_t i = 0; i < theSize ; i++) { vec[i].inc(); } " << timer.elapsed() << "\n"; } I get following times when run on suse linux with gcc 4.2.1/ boost 1.39 Time: BOOST_FOREACH(Fred& fred, vec) { fred.inc(); } 10.65 Time: std::for_each(vec.begin(),vec.end(),boost::bind(&Fred::inc,_1) ); 9.76 Time: for (std::vector<Fred>::iterator i = vec.begin(); i < theEnd ; i++) { (*i).inc(); } 4.48 Time: std::for_each(vec.begin();vec.end(),std::mem_fun_ref(&Fred::inc)) 4.08 Time: for (size_t i = 0; i < theSize ; i++) { vec[i].inc(); } 1.6 It appears that BOOST_FOREACH and use of boost::bind in performance critical application should be avoided ? -- Best regards, Ta, Avi

2 1

Problem with Interprocess Synchronization MUTEX
by Daniel Veneros 05 Feb '10

05 Feb '10

Hi, I have a problem, I need to access a shared memory segment from several process and all at the same time. I was looking at boost::interprocess threads but I'm not sure how to implement the mutex. this is the code I have(working): #include <boost/interprocess/managed_shared_memory.hpp> #include <boost/interprocess/containers/vector.hpp> #include <boost/interprocess/containers/string.hpp> #include <boost/interprocess/containers/map.hpp> #include <boost/interprocess/allocators/allocator.hpp> #include <cstdlib> #include <string> #include <iostream> #include <map> #include <sstream> using namespace boost::interprocess; typedef managed_shared_memory::segment_manager SegmentManager; typedef allocator<void, SegmentManager> VoidAllocator; typedef allocator<char, SegmentManager> CharAllocator; typedef basic_string<char, std::char_traits<char>, CharAllocator> BasicString; class Action { int id; BasicString task; public: Action(int num, const char *name, const VoidAllocator &void_alloc) :id(num), task(name,void_alloc) {} void setId(int num); void setTask(BasicString newTask); int getId(); }; int Action::getId() { return id; } void Action::setId(int num) { id = num; } void Action::setTask(BasicString newTask) { task = newTask; } typedef allocator<Action, SegmentManager> ActionAllocator; typedef std::pair<const BasicString, Action> MapValueType; typedef std::pair<BasicString, Action> MovableToMapValueType; typedef allocator<MapValueType, SegmentManager> MapValueTypeAllocator; typedef map<BasicString, Action, std::less<BasicString>, MapValueTypeAllocator> MyMap; void CreateAndFill() { managed_shared_memory segment(create_only, "MySharedMemory", 65536); VoidAllocator alloc_inst(segment.get_segment_manager()); MyMap *myMap = segment.construct<MyMap>("Map" )(std::less<BasicString>(),alloc_inst); std::string keyword; std::string helper; int i=0; for(int adder = 0; adder<=1000; adder++) { keyword = "Hello"; std::stringstream out; out << adder; helper = out.str(); keyword = keyword + helper;//Creates a string "Hello" + adder BasicString key(keyword.c_str(), alloc_inst);//string to BasicString Action action(i,"ActionContent", alloc_inst);//puts values of Action object MapValueType value(key, action); myMap->insert(value); i++; } } int main(int argc, char** argv) { //Remove shared memory on construction and destruction struct shm_remove { shm_remove() {shared_memory_object::remove("MySharedMemory"); } ~shm_remove() {shared_memory_object::remove("MySharedMemory"); } }remover; CreateAndFill(); } How and where do I have to set the mutex?? Is this anonymous or named? Thanks! Dann

1 0

[StateChart] Inheriting state
by Igor R 05 Feb '10

05 Feb '10

Hello, I'd like to share some functionality between states belonging to different (but similar) FSMs. I try to do this by taking out some part of a state to a base class template: template<class Derived, class Outer, class Inner> struct ActiveBase : state<Derived, Outer, Inner> { // this's probably not quite standard, but it should be fine with msvc ActiveBase(my_context ctx) : my_base(ctx) {} void f(const Event1 &) {} }; struct Active; struct FSM : asynchronous_state_machine<FSM, Active> { //... }; struct Active : ActiveBase<Active, FSM, Disconnected> { Active(my_context ctx) : ActiveBase(ctx) {} typedef mpl::list < sc::in_state_reaction<Event1, ActiveBase, &ActiveBase::f> > reactions; }; However, this doesn't compile: 1>..\boost/statechart/state_machine.hpp(493) : error C2440: 'return' : cannot convert from 'FSM' to 'ActiveBase<Derived,Outer,Inner> &' 1> with 1> [ 1> Derived=Active, 1> Outer=FSM, 1> Inner=boost::mpl::list<Disconnected> 1> ] 1> ../boost/statechart/simple_state.hpp(684) : see reference to function template instantiation 'Context &boost::statechart::state_machine<MostDerived,InitialState,Allocator,ExceptionTranslator>::context<OtherContext>(void)' being compiled 1> with 1> [ 1> Context=ActiveBase<Active,FSM,boost::mpl::list<Disconnected>>, 1> MostDerived=FSM, 1> InitialState=Active, 1> Allocator=std::allocator<void>, 1> ExceptionTranslator=boost::statechart::null_exception_translator, 1> OtherContext=ActiveBase<Active,FSM,boost::mpl::list<Disconnected>> 1> ] etc........ Of course, the above snippet is not the real code, and maybe I miss the problem in some other place, but I just would like to know if the above approach legitimate and *should* compile. Thanks.

2 7

Multicast Dual NIC Receive Problem
by Andy Couldrake 05 Feb '10

05 Feb '10

As a newbie to Boost asio, can please I ask for some guidance on the following problem (which does seem to get a mention from time to time on the boost lists). On a dual NIC Linux server, I am unable to get the sample boost asio multicast receiver ( http://www.boost.org/doc/libs/1_41_0/doc/html/boost_asio/example/multicast/…) to receive packets. It does work fine on a single NIC workstation running the same version of Linux by specify 0.0.0.0 as the listen address, but when I specify the listen address of the correct card on the dual NIC server (A.B.C.D), I get nothing. I have also tried the following but no luck: boost::asio::ip::multicast::join_group joinRequest(multicast_address.to_v4(), listen_address.to_v4()); socket_.set_option(joinRequest); As a sanity check: mtools mdump (http://www.29west.com/news/mtools/) shows that multicast data is coming through ok on the dual NIC server on the NIC listenaddress (A.B.C.D) eg ./mdump <groupaddress> <port> <listenaddress>. It would be nice if we could get the sample code corrected and working! I volunteer to help achieve this ... Regards, Andy

1 0

Re: [Boost-users] [Serialization] Speedding up client-server communication
by Brian Wood 05 Feb '10

05 Feb '10

Ruediger Berlich writes: > Hi there, > > I am in the process of speeding up communication between a server and its > clients. Communication involves serialized class data. Messages can be as > large as 100 kilobytes. > I have done some measurements which have shown that, in a cluster with > Gigabit networking, most overhead of the parallelisation seems to come from > the Broker infrastructure and the process of (de-)serialization. Network > latency and/or bandwidth seems to play only a minor role in this > environment. > Hence, apart from optimizing my broker, I'm looking for ways to optimize the > serialization process, as used in my application. As messages are discarded > as soon as they reach the recipient, versions of serialized data do not play > an important role. I don't find any mention of "message" in the 1.42 Boost Serialization documentation. Are you using MPI? The obvious thing you've not mentioned is compression. I use bzip2 to compress and decompress data - http://webEbenezer.net/misc/SendCompressedBuffer.hh http://webEbenezer.net/misc/ReceiveCompressedBuffer.hh Besides using bzip2, I use variable-length integers to encode the size of the compressed data. It works to use a fixed-size integer as well, but frequently you can shave a couple of bytes off the total by using a variable-length integer. Brian Wood http://webEbenezer.net (651) 251-9384 "The kingdom of heaven is like a treasure hidden in the field, which a man found and hid again; and from joy over it he goes and sells all that he has and buys that field."

1 0

problem with browsing html documentation under w3m
by Michal 04 Feb '10

04 Feb '10

Hallo Group Members there is one page that causes problems under w3m: http://www.boost.org/doc/libs/1_39_0/libs/range/doc/range.html <link href="style.css" rel="stylesheet" type="text/css"><link href="/favicon.ico" rel="icon" type="image/ico"><link href="/style/section-basic.css" rel="stylesheet" type="text/css"><pre_int><img_alt src="../../../boost.png" hseq="1">[boost] </img_alt></pre_int> Boost.Range Range concepts <_SYMBOL TYPE=32>•</_SYMBOL> Overview <_SYMBOL TYPE=32>•</_SYMBOL> Single Pass Range <_SYMBOL TYPE=32>•</_SYMBOL> Forward Range <_SYMBOL TYPE=32>•</_SYMBOL> Bidirectional Range <_SYMBOL TYPE=32>•</_SYMBOL> Random Access Range <_SYMBOL TYPE=32>•</_SYMBOL> Concept Checking <nobr><_SYMBOL TYPE=26>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━</_SYMBOL> </nobr> Overview A Range is a concept similar to the STL Container concept. A Range provides iterators for accessing a half-open range [first,one_past_last) of elements and provides information about the number of elements in the Range. However, a Range has much fewer requirements than a Container. The motivation for the Range concept is that there are many useful Container-like types that do not meet the full requirements of Container, and many algorithms that can be written with this reduced set of requirements. In particular, a Range does not necessarily <_SYMBOL TYPE=32>•</_SYMBOL> own the elements that can be accessed through it, <_SYMBOL TYPE=32>•</_SYMBOL> have copy semantics, best regards, Michal

3 2

PP: 2 to the power x macro
by Hicham Mouline 04 Feb '10

04 Feb '10

Hello, I have written this code to return 2 to the power x. #include <boost/preprocessor/repetition/for.hpp> #include <boost/preprocessor/seq/size.hpp> #include <boost/preprocessor/arithmetic/add.hpp> #include <boost/preprocessor/arithmetic/sub.hpp> #include <boost/preprocessor/arithmetic/mul.hpp> #include <boost/preprocessor/tuple/elem.hpp> #include <boost/preprocessor/seq/elem.hpp> #define PRED(r,state) BOOST_PP_TUPLE_ELEM(2, 1, state) #define OP(r,state) (BOOST_PP_MUL(BOOST_PP_TUPLE_ELEM(2, 0, state), 2), BOOST_PP_SUB(BOOST_PP_TUPLE_ELEM(2, 1, state), 1)) #define MACRO(r,state) (state) #define POWERSEQ(x) BOOST_PP_FOR((2,x), PRED, OP, MACRO) #define LAST_TUPLE(x) BOOST_PP_SEQ_ELEM(BOOST_PP_SUB(BOOST_PP_SEQ_SIZE(POWERSEQ(x)),1), POWERSEQ(x)) #define POWER2(x) BOOST_PP_TUPLE_ELEM(2, 0, LAST_TUPLE(x)) POWER2(6) I have to generate some sequence and take the last element of it and then extract the result. I didn't find such a macro already in PP. My code seems a bit too complicated. Is there a more trivial way to do this? rds,

3 7

Using Boost 1.41 / output directories
by Rocks User 04 Feb '10

04 Feb '10

Hello, We are migrating to Boost 1.41.0 from boost 1.37.0. When using 1.41.0, the output path for the object files is different than when using 1.37.0. For ex: source code: home/proj/moduleA/1 home/proj/moduleA/2 home/proj/moduleA/3 All object files from moduleA/1, /2, /3 are as follows, when using 1.37.0 home/proj/build/gcc/threading/debug/moduleA/ When using 1.41.0, the object files are created as follows: home/build/gcc/threading/debug/moduleA/1 home/build/gcc/threading/debug/moduleA/2 home/build/gcc/threading/debug/moduleA/3 This is messing up our build esp the header file include paths becasue some of out .h & .cpp files are dynamically generated to the build location and compiled How can I make the object files layout similar to when using 1.37.0? Any help is highly appreciated. Thanks,

2 1

Re: [Boost-users] Help with custom classes PLEASE!!
by Steven Watanabe 04 Feb '10

04 Feb '10

AMDG Daniel Veneros wrote: > Thanks, > > I fixed some things and its running now. > What I don't know how to do is to declarate/allocate a custom class.. > I got this: > > typedef managed_shared_memory::segment_manager SegmentManager; > > typedef allocator<char, SegmentManager> CharAllocator; > > typedef basic_string<char, std::char_traits<char>, CharAllocator> > BasicString; > > typedef allocator<int, SegmentManager> IntAllocator; > > > class Action > { > int id; > BasicString task; > public: > Action(int num, const char *name, const VoidAllocator &void_alloc) > :id(num), task(name,void_alloc) > {} > void setId(int num); > void setTask(BasicString newTask); > }; > > void Action::setId(int num) { > id = num; > } > > void Action::setTask(BasicString newTask) { > task = newTask; > } > > > Now... how should I allocate the "Class" object?? > Any Idea? Thanks!! > I assume that by Class you mean the Action class? If this class is used at the top level, you can use managed_shared_memory::construct. If you want to use an allocator with it, the commands are allocator<Action, SegmentManager> alloc(...); // allocate memory Action* action = alloc.allocate(1); // construct an object using placement new new (action) Action(12, "action", ...); In Christ, Steven Watanabe P.S. Please send messages like this to boost-users rather than directly to me.

2 1