- Boost - lists.boost.org

Re: [boost] asio formal review begins
by christopher baus 16 Dec '05

16 Dec '05

> What I did was to have the main thread accept incomming requests, and put > them in the queue, whereas a number of worker threads was taking these > requests from the queue, and execute them. I don't see how this model can > be absolutely inferior in all possible contexts. > I think your argument regarding the sync interface *might* be valid for other reasons, but I want to point out that this architecture only scales if you have an async front end that reads and writes from the socket and parses the connection into tasks in the main thread, and then handles these tasks synchronously in a thread pool. Say for instance a web server which handles HTTP request pipelines asynchronously and database connections synchronously. If the system contains long lived connections, then you are basically only servicing the number of connections in your thread pool, and there is a good chance the other connections will timeout. I do agree with Peter. If asio leads developers away from thread/connection architectures, then it has succeeded. As with a lot of technologies that are shiny and new, I think the industry has put too much emphasis on threads as the solution to all concurrency problems. I think we understand the problems better now. Christopher

4 4

asio review: are we still on topic?
by Arkadiy Vertleyb 16 Dec '05

16 Dec '05

Guys, This general discussion about which approach to networking is superior is very interesting (and useful for me personally), but I think we moved off topic. I think Chris (the author of asio) agreed that both approaches are valid, and claimed he gives both of them appropriate attention. So now, since I pointed to what I believe is a problem with the synchronous interface in asio (socket dependency upon the demuxer), doesn't the critique of synchronous approach sounds a little bit like: "It is inferior anyway, so it's a goot thing that it's not clean" ? Because nobody yet tried to convince me that socket dependency upon demuxer (even if for synchronous IO only) is a good thing. Regards, Arkadiy

5 4

[filesystem] Additional Decomposition Request
by me22 16 Dec '05

16 Dec '05

Hi all, The discussion a little bit ago regarding using path's grammar in a custom virtual filesystem reminded me of my experience when I made a simple unpacking program using boost::filesystem::path. One thing that wasn't as trivial as I might have liked was descending recursively down the internal hierarchy. It would have been nice to have head() and tail() decomposition functions ( though with possibly better names ) analogous to the branch_path() and leaf() functions, but for the front of the path. That way I could have simply looked for the head() element at the current level and recursed with tail(). ( Though I ended up doing it by having my get_file function delegate to one that worked with an iterator range, which is more efficient, but... ) The implementation is, of course, fairly trivial, something along the lines of the following: std::string path::head() { if ( empty() ) return std::string(); return *begin(); } path path::tail() { if ( empty() ) return path(); return std::accumulate( next(begin()), end(), std::divides<path>() ); } And the obvious has_head() and has_tail(). The usage would be something similar to: file &directory::get_file(boost::filesystem::path p) { if ( !p.has_tail() ) return files[ p.head() ]; return subdirectories[ p.head() ].get_file( p.tail() ); } Assuming appropriate ( not part of boost ) directory and file classes. Opinions? Interest? - Scott McMurray P.S. I'm particularly fond of the accumulate/divides hack :P

1 0

[serialization][bug]
by Oleg Abrosimov 16 Dec '05

16 Dec '05

Hello! Some days ago I've faced with very strange problem with boost::serialization it doesn't compile, though docs says that it should. After some research I've found that changing the order of header files included fixes the problem. It can be illustrated on a base of the demo.cpp program included in distribution: The original order: #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> #include <boost/serialization/base_object.hpp> #include <boost/serialization/utility.hpp> #include <boost/serialization/list.hpp> The modified order [doesn't compile!]: #include <boost/serialization/base_object.hpp> #include <boost/serialization/utility.hpp> #include <boost/serialization/list.hpp> #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> It is crucial to include these files first #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> it is because of many files (may be indirectly) include boost/serialization/type_info_implementation.hpp header that uses BOOST_SERIALIZATION_DEFAULT_TYPE_INFO macro without defining it, and #include <boost/archive/text_iarchive.hpp> #include <boost/archive/text_oarchive.hpp> headers define this macro in some way. to conclude: I have found it very disappointing that changing of order of included headers changes code correctness so dramatically. I want it to be fixed in some way. Thanks for great and incredibly helpful library anyway, Oleg Abrosimov. PS: The compiler (VC7.1) output: Compiling... demo.cpp D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(131) : error C2027: use of undefined type 'boost::serialization::extended_type_info_null<T>' with [ T=bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(131) : see reference to class template instantiation 'boost::serialization::extended_type_info_null<T>' being compiled with [ T=bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(128) : while compiling class-template member function 'bool boost::archive::detail::oserializer<Archive,T>::is_polymorphic(void) const' with [ Archive=boost::archive::text_oarchive, T=bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(264) : see reference to class template instantiation 'boost::archive::detail::oserializer<Archive,T>' being compiled with [ Archive=boost::archive::text_oarchive, T=bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(263) : while compiling class-template member function 'void boost::archive::detail::save_non_pointer_type<Archive,T>::save_standard::invoke(Archive &,const bus_schedule &)' with [ Archive=boost::archive::text_oarchive, T=bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(322) : see reference to class template instantiation 'boost::archive::detail::save_non_pointer_type<Archive,T>::save_standard' being compiled with [ Archive=boost::archive::text_oarchive, T=bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(310) : while compiling class-template member function 'void boost::archive::detail::save_non_pointer_type<Archive,T>::invoke(Archive &,const bus_schedule &)' with [ Archive=boost::archive::text_oarchive, T=bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(536) : see reference to class template instantiation 'boost::archive::detail::save_non_pointer_type<Archive,T>' being compiled with [ Archive=boost::archive::text_oarchive, T=bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\basic_text_oarchive.hpp(78) : see reference to function template instantiation 'void boost::archive::save<Archive,T>(Archive &,T &)' being compiled with [ Archive=boost::archive::text_oarchive, T=const bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\interface_oarchive.hpp(78) : see reference to function template instantiation 'void boost::archive::basic_text_oarchive<Archive>::save_override<T>(T &,int)' being compiled with [ Archive=boost::archive::text_oarchive, T=const bus_schedule ] demo.cpp(286) : see reference to function template instantiation 'Archive &boost::archive::detail::interface_oarchive<Archive>::operator <<<const bus_schedule>(T &)' being compiled with [ Archive=boost::archive::text_oarchive, T=const bus_schedule ] D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(131) : error C2146: syntax error : missing ';' before identifier 'typex' D:\home\olegabr\devel\boost_1_33_1\boost\archive\detail\oserializer.hpp(131) : error C2065: 'typex' : undeclared identifier

2 1

Re: [boost] asio formal review begins
by christopher baus 16 Dec '05

16 Dec '05

> I happen to > think that the fact many people have been trained to think about sockets synchronously is most definitely for the worse. Totally agree >> I just want to point out that async reading and writing could >> be a function of an async I/O demuxer and not the socket >> itself. The socket could be passed to the demuxer and not >> vice versa. > The above design is something I can fundamentally disagree with > ;) That sort of design, in my opinion, relegates asynchronous > operations to "second class citizens". I'm not religious about this, but it could be argued that it would put sync and async on the same level. For instance you could consider the POSIX interface to be os::sync_io::write(fd, buffer, bytes) and the os::sync_io is just left out. Then my interface would be os::async_demuxer::write(fd, buffer, bytes); But honestly this isn't something that concerns me. > There is a relatively straightforward mapping between one and > the other. In the future I want to exploit this mapping further > by investigating the use of expression templates (perhaps > similar to Boost.Lambda) to permit the encoding of a sequence of > operations in a synchronous programming style. The operations > could then be executed either synchronously or asynchronously as needed. Not to take away from asio (which I think is great BTW), but before jumping into discussion about expression templates, I want to point out that accepting handlers as template parameters and passing them by value isn't free in asio. The end result is a dynamic copy of the handler per operation in a system with potentially thousands of operations/second. This is going to make potential pooling inefficient because the library doesn't know what size and how many handlers are used until run time (what happens if one large handler is queued along with a bunch of small ones?). The handler functions are called via a function pointer, so there isn't any performance advantage by passing handler functors by value, and the deferred nature of async i/o makes compile time binding impossible (ok I shouldn't say impossible with this group, but extremely unlikely). I'm going to discuss this more in my review, but there is a significant amount of overhead that is an artifact of the value semantics. I think a parallel interface which accepts handlers by reference (or pointer) should be considered. The result would be that the user could specify the same handler instance for multiple operations, and asio could pool fixed sized queue nodes. I suspect that in applications handling a large number of connections the result would be improved reliability, performance, and in some cases lower memory usage. Anybody that's interested in the *nix implementation of this should look at detail/reactor_op_queue.hpp, which is an interesting read. The allocation I'm discussing happens in enqueue_operation(). Similar allocations happen in win_iocp_socket_service.hpp and win_iocp_demuxer_service. I think the allocation for value functors could be done in a cross platform area of the library, and a concrete handler object pointer could be passed to platform specific implementations which allocate fixed sized structures from pools. Other than that concern with memory management, I become more impressed by the library the more time I spend with it. It is really a great piece of work. Cheers, Christopher

1 0

[filesystem] problem with filename length (bug?)
by Allmen, Michael-P57512 16 Dec '05

16 Dec '05

Here is the code I am using (the problem also occurs with the simple_ls.cpp sample)... #include <iostream> #include "boost/filesystem/operations.hpp" #include "boost/filesystem/path.hpp" namespace fs = boost::filesystem; int main( int argc, char* argv[] ) { fs::path dir( "somedir" ); fs::directory_iterator end; for( fs::directory_iterator i( dir ); i != end; ++i ) { std::cout << dir->leaf() << std::endl; } return 0; } The problem is that once the program finds a file at least 22 characters long, it will not print any filenames longer than first file correctly. In "somedir" are several long filenames. Here is the contents of the directory in the order the files were created: somereallyreallylongfilename somereallylongfilename someotherreallylongfilename somereallyreallylongfilenam3 somereallyreallylongfilenames The program outputs: somereallyreallylongfilename somereallylongfilename someotherreallylongfilename somereallyreallylongfilenam3 somereallyreallylongfi (here the output stops and goes back to my prompt) Anyone else had this problem? Thanks, Mike

2 1

asio async call terminology
by christopher baus 16 Dec '05

16 Dec '05

I'm working on my review and I want to make sure I am using the correct terminology. There two sorts of asynchronous calls that can be made. The first is a "operation" which results in one trip through the dispatcher. For instance asio::socket_stream::write_some(). Internally these are stored in operation classes, hence the terminology. The second I'm calling a "request" which results in one or more "operations" to occur before the user handler is dispatched. For instance asio::async_write() calls asio::stream_socket::write_some() one or more times. Does that sound right?

2 1

[Boost.Filesystem] Feature request
by Alex Besogonov 15 Dec '05

15 Dec '05

Good <time-of-day>! It would be nice to have a utility for walking the filesystem tree. Naive recursive implementation will not work because there may be cycles in filesystem (due to hard/soft links and 'map' feature of 'mount'). PS: What's the status of i18ned filesystem library? -- With respect, Alex Besogonov (cyberax(a)elewise.com)

2 1

[Test] can't build boost.test dll's on MacOSX
by Kim Barrett 15 Dec '05

15 Dec '05

When building Boost_1_33_1 on MacOSX 10.4 with Darwin tools (gcc 4.0.1), if I try to build with dynamic linking enabled (i.e. "<runtime-link>dynamic" in the bjam command), there are link-time failures for the three libraries that are part of Boost.Test. I think this is likely the same as a known problem with NT mentioned in recent email from Gennadiy: the prg_exec_monitor and test_exec_monitor libraries fail because __Z9test_mainiPPc is undefined, while the unit_test_framework library fails due to __Z20init_unit_test_suiteiPPc being undefined. It looks like the Jamfile for Boost.Test has a workaround for NT here, which presumably needs to be extended to MacOSX as well. There is a block of code in libs/test/build/Jamfile that looks like this: local TEST_DLLs ; if ! $(NT) && ( ! $(UNIX) || $(OS) != CYGWIN ) { TEST_DLLs = TRUE ; } It looks like the conditional there needs to also be false for MacOSX. After grovelling around in the jam configuration, it looked like the necessary change was to replace $(OS) != CYGWIN with ( $(OS) != CYGWIN && $(OS) != MACOSX ) i.e. the test should be if ! $(NT) && ( ! $(UNIX) || ( $(OS) != CYGWIN && $(OS) != MACOSX )) Unfortunately, that change doesn't seem to be sufficient to disable the attempt to build dll's for Boost.Test. I have no idea why, and no idea how to figure that out, being largely unfamiliar with jam and Jamfiles. Alternatively, is there a way from the bjam command to indicate that I want <runtime-link>dynamic except for some specific set of libraries? (I know how to exclude libraries completely, which would probably solve my problem for prg_exec_monitor and test_exec_monitor, since we aren't using them, but doesn't deal with the unit_test_framework library, which I do need.) It would seem from Gennadiy's recent burst of messages that this may no longer a problem or about to no longer be a problem on CVS HEAD, but we're not prepared to go there and would prefer a patch to the 1.33.1 release. Sorry I didn't notice this when checking out the release candidate, but this platform has only just been bumped up in my priorities enough for me to start trying it.

2 2

Re: [boost] Error in new boost.test
by Martin Bonner 15 Dec '05

15 Dec '05

----Original Message---- From: Markus Schöpflin [mailto:markus.schoepflin@comsoft.de] > (std::min)(it->m_alloc.size, (unsigned)8 ) > I think the (unsigned)8 should actually read > static_cast<std::size_t>(8), > or perhaps just use (std:min<std::size_t>) instead. I presume the parens around std::min are to prevent VC expanding the "min" macro. If so, then adding the expliction specialization should do that anyway, so the code can read: std::min<std::size_t>(it->m_alloc.size, 8 ) Which remarkably is only one character longer than the unfixed code (and doesn't have a c-style cast). -- Martin Bonner Martin.Bonner(a)Pitechnology.com Pi Technology, Milton Hall, Ely Road, Milton, Cambridge, CB4 6WZ, ENGLAND Tel: +44 (0)1223 441434

2 1