October 2007 - Boost - lists.boost.org

bad include guards
by Steven Watanabe 26 Jan '17

26 Jan '17

AMDG The following pairs of files have identical include guards ./boost/concept_check/general.hpp ./boost/concept/detail/msvc.hpp ./boost/concept_check/msvc.hpp ./boost/concept/detail/msvc.hpp ./boost/filesystem/cerrno.hpp ./boost/cerrno.hpp ./boost/interprocess/containers/tree.hpp ./boost/interprocess/containers/detail/tree.hpp ./boost/mpl/min_max.hpp ./boost/mpl/max.hpp ./boost/mpl/vector/aux_/front.hpp ./boost/mpl/list/aux_/front.hpp ./boost/multi_index/identity_fwd.hpp ./boost/multi_index/identity.hpp ./boost/python/pointee.hpp ./boost/python/detail/pointee.hpp ./boost/spirit/phoenix/closures.hpp ./boost/lambda/closures.hpp ./boost/xpressive/proto/v1_/proto_typeof.hpp ./boost/xpressive/proto/proto_typeof.hpp ./libs/config/test/boost_no_dep_nested_class.ipp ./libs/config/test/boost_has_vc6_mem_templ.ipp ./libs/config/test/boost_no_mem_templates.ipp ./libs/config/test/boost_has_vc6_mem_templ.ipp ./libs/config/test/boost_no_std_allocator.ipp ./libs/config/test/boost_has_vc6_mem_templ.ipp ./libs/conversion/test.hpp ./libs/any/test.hpp ./libs/interprocess/example/print_container.hpp ./libs/interprocess/example/printcontainer.hpp ./libs/interprocess/test/expand_bwd_test_template.hpp ./libs/interprocess/test/allocation_test_template.hpp ./libs/interprocess/test/printcontainer.hpp ./libs/interprocess/example/printcontainer.hpp ./libs/interprocess/test/print_container.hpp ./libs/interprocess/example/printcontainer.hpp ./libs/iostreams/test/detail/utf8_codecvt_facet.hpp ./boost/detail/utf8_codecvt_facet.hpp ./libs/regex/test/regress/test_regex_search.hpp ./libs/regex/test/regress/test_partial_match.hpp ./libs/serialization/test/C.hpp ./libs/serialization/test/A.hpp ./libs/xpressive/perf/regex_comparison.hpp ./libs/regex/performance/regex_comparison.hpp ./tools/inspect/time_string.hpp ./tools/common/time_string.hpp In Christ, Steven Watanabe

5 8

Proposed addition to the iterator library for iterating over all tuples.
by Alex Mendes da Costa 19 Jun '09

19 Jun '09

Hi folks: I quite often have to write code to do the following: Given N sequences, generate all the N-tuples s.t. the i'th element in each tuple is drawn from the i'th sequence. For example, if sequence S1 is { 1, 2, 3 } and sequence S2 is { 10, 20 }, I need to generate these tuples: (1, 10), (2, 10), (1, 20), (2, 20). I think this problem could be solved elegantly with a iterator similar to zip_iterator. I have some questions: (1) Is there already a way to do this in boost? (2) If not, would this be a worthwhile addition to the iterator library? (3) If so, could I contribute code for this, or must the existing library owners write it? (4) If so, how would I go about getting my code added? Most of the docs seem to be about submitting entire libraries, rather than extensions to existing libraries. Thanks, Alex

4 18

[asio]: extending asio
by Andreas Pokorny 17 Aug '08

17 Aug '08

Hello, I would like to add more asynchronous event sources to asio, like: - input events for mouse, keyboard and joystick .. - window exposure, move, resize events in some generic form, - generic "quit" events - file alteration With directFB/X11/Unix/Linux most of that can be done directly through select or epoll. But since reading the message and parsing events is mostly done by a third party library, I would only need the reactor part for these. Is this possible with asio? According to msdn it seems to be possible to create so called event objects for direct input devices. I am no win32 expert so I do not know if one can combine that with completion ports? Regards Andreas Pokorny

7 16

Review Request:boost.lexer
by Ben Hanson 01 Feb '08

01 Feb '08

Hi There, I have posted boost.lexer to the Boost File Vault and would like to request a review. Hartmut has integrated my library 'lexertl' into Spirit already, so this is the boost version of that library. Thanks, Ben Hanson

3 2

Strings tagged with their character set
by Phil Endecott 10 Jan '08

10 Jan '08

Dear All, Something that I have been thinking about for a while is storing strings tagged with their character set. Since I now have a practical need for this I plan to try to implement something. Your feedback would be appreciated. The starting point is the idea that the character set of a string may be known at compile time or at run time, and so two types of tagging are possible. First compile-time tagging: template <character_set> class tagged_string { ... }; tagged_string<utf8> s1; tagged_string<latin1> s2; Some typedefs would be appropriate: typedef tagged_string<utf8> utf8string; Now run-time tagging: class rt_tagged_string { private: character_set cs; public: rt_tagged_string(character_set cs_): cs(cs_) ... ... }; rt_tagged_string(utf8) s3; (Consise-yet-clear names for any of these classes would be great.) I propose to implement conversion between the strings using icode and/or GNU recode. It would be easy to allow this conversion to happen invisibly, but it might be wiser to make conversion explicit. I'm not sure what the 'character_set' that I've used above should be. It needs to be some sort of user-extensible enum or type-tag. We need character types of 8, 16 and 32 bits. wchar is not useful here because it's not defined whether it's 16 or 32 bits. So I propose the following, modelled after cstdint: typedef char char8_t; typedef <implementation-defined> char16_t; typedef <implementation-defined> char32_t; I then propose a character_set_traits class: template <character_set> class character_set_traits; template <> class character_set_traits<utf8> { typdef char8_t char_t; const bool variable_width = true; ... }; For the fixed-width, compile-time-tagged strings I think it makes sense to inherit from std::basic_string< character_set_traits<charset>::char_t >. The only problem I can see with this is that latin1string s1 = "hello world"; s1.substr(1,5) <--- this returns a std::string, not a latin1string If latin1string has a constructor from std::string (which is its own base type) that's fine, i.e. we can still write: latin1string s2 = s1.substr(1,5); but unfortunately we can also write latin2string s3 = s1.substr(1,5); which is not so good. So a different approach is to define a set of character-set-specific character types, and build string types from them: typedef char8_t latin1char; typedef char8_t latin2char; For variable-width character sets, the methods of std::string are less useful (though far from useless). I understand that there's already a utf8 iterator somewhere in Boost, can it help? For run-time character sets, is there any way to provide e.g. run-time iterators? I imagine these strings being used as follows: - Input to the program is either run-time or compile-time tagged with any character set. - Data that is not manipulated in any way it just passed through. - Data that will be processed will first be converted to a suitable, compile-time-tagged, character set, and if appropriate converted back afterwards. So the absence of (useful) string operations on run-time-tagged or variable-width character set data is not a problem. For conversions, there is the question of partial characters in variable-width character sets. If a program is processing data in chunks it may be legitimate for a chunk boundary to fall in the middle of a UTF8 character. IIRC, icode has a method to deal with this which we could expose in a stateful converter: charset_converter utf8_to_ucs4(utf8,ucs4); while (!eof) { utf8string s = get_chunk(); ucs4string t = utf8_to_ucs4(s); send_chunk(t); } utf8_to_ucs4.flush(); - but many applications may only need a stateless converter. I will be working on this over the next couple of weeks, so any feedback would be much appreciated. Regards, Phil.

16 55

seek issue with boost::iostreams::stream
by Steve Nutt 08 Jan '08

08 Jan '08

I've created a device with the boost::iostreams::dual_seekable category to use with the boost::iostreams::stream class, but I'm having trouble implementing seek. When I call seekp on the stream, seek on my device is called with a which (openmode) of both BOOST_IOS::in and BOOST_IOS::out. I would have expected for a call to seekp the which parameter in the seek call on my device would be BOOST_IOS::out, and calling seekg would set which to BOOST_IOS::in. The following code in boost\iostream\detail\streambuf\indirect_streambuf.hpp ignores the which argument passed and passes on through both in and out. template<typename T, typename Tr, typename Alloc, typename Mode> inline typename indirect_streambuf<T, Tr, Alloc, Mode>::pos_type indirect_streambuf<T, Tr, Alloc, Mode>::seekpos (pos_type sp, BOOST_IOS::openmode) { return seek_impl( position_to_offset(sp), BOOST_IOS::beg, BOOST_IOS::in | BOOST_IOS::out ); } Is this correct, and if so how do I implement seeking when using a dual head device? Many thanks, Steve Nutt

2 2

[IOStreams] Opinion on test case needed
by Frank Birbacher 06 Jan '08

06 Jan '08

Hi! I had problems using iostreams::stream<file_descriptor> with Windows and MSVC 8 Express. The file_descriptor had a bug and failed seeking. But the error had gone unnoticed. Because of this I wrote a test case for this in hope to discover an error in IOStreams (source: see below). I did not discover one. Instead the test runs fine on linux/gcc 4.1.2 but fails on Windows/MSVC8Express (test run: see below). I based the test on "Standard C++ IOStreams and Locales" (Langer&Kreft), Section 1.3 "The Stream State". And I found the IOStreams library is just using std::iostream for handling the errors. So is MSVC buggy in this regard? Or am I missing something? Source of test case in Trac SVN Browser: http://svn.boost.org/trac/boost/browser/sandbox-branches/birbacher/fix_iost… Result of test case on platform Windows/MSVC 8 Express: > ...using <p..\..\..\bin.v2\libs\iostreams\test\error_test.test\msvc-8.0express\debug>error_test.exe... > testing.capture-output ..\..\..\bin.v2\libs\iostreams\test\error_test.test\msvc-8.0express\debug\error_test.run > ====== BEGIN OUTPUT ====== > Running 12 test cases... > error_test.cpp(81): error in "wrap_nothrow <&test_seekg>": exception thrown by function(stream) > error_test.cpp(66): error in "wrap_nothrow <&test_seekg>": stream still good > error_test.cpp(68): error in "wrap_nothrow <&test_seekg>": stream did not set badbit > error_test.cpp(69): error in "wrap_nothrow <&test_seekg>": stream did not fail > error_test.cpp(71): error in "wrap_nothrow <&test_seekg>": stream does not report failure by operator ! > error_test.cpp(73): error in "wrap_nothrow <&test_seekg>": stream does not report failure by operator void* > error_test.cpp(66): error in "wrap_throw <&test_seekg>": stream still good > error_test.cpp(68): error in "wrap_throw <&test_seekg>": stream did not set badbit > error_test.cpp(69): error in "wrap_throw <&test_seekg>": stream did not fail > error_test.cpp(71): error in "wrap_throw <&test_seekg>": stream does not report failure by operator ! > error_test.cpp(73): error in "wrap_throw <&test_seekg>": stream does not report failure by operator void* > unknown location(0): fatal error in "wrap_throw_delayed<&test_seekg>": std::runtime_error: bad seek > error_test.cpp(73): last checkpoint > error_test.cpp(81): error in "wrap_nothrow <&test_seekp>": exception thrown by function(stream) > error_test.cpp(66): error in "wrap_nothrow <&test_seekp>": stream still good > error_test.cpp(68): error in "wrap_nothrow <&test_seekp>": stream did not set badbit > error_test.cpp(69): error in "wrap_nothrow <&test_seekp>": stream did not fail > error_test.cpp(71): error in "wrap_nothrow <&test_seekp>": stream does not report failure by operator ! > error_test.cpp(73): error in "wrap_nothrow <&test_seekp>": stream does not report failure by operator void* > error_test.cpp(66): error in "wrap_throw <&test_seekp>": stream still good > error_test.cpp(68): error in "wrap_throw <&test_seekp>": stream did not set badbit > error_test.cpp(69): error in "wrap_throw <&test_seekp>": stream did not fail > error_test.cpp(71): error in "wrap_throw <&test_seekp>": stream does not report failure by operator ! > error_test.cpp(73): error in "wrap_throw <&test_seekp>": stream does not report failure by operator void* > unknown location(0): fatal error in "wrap_throw_delayed<&test_seekp>": std::runtime_error: bad seek > error_test.cpp(73): last checkpoint > > *** 24 failures detected in test suite "Master Test Suite" > > EXIT STATUS: 201 > ====== END OUTPUT ====== Any opinions on this? Any suggestions on what to do about it? Frank

4 16

[iostreams] understanding the models
by Benjamin Collins 05 Jan '08

05 Jan '08

I'm trying to write a device for iostreams, but I'm having a hard time understanding how the current sources and the models explained in the documentation are in agreement. Maybe I'm shooting myself in the foot by never being able to work on this for more than 30 minutes at a time and having that 30 minutes be late at night, but... Anyway, it seems odd to me - for example, the boost::iostreams::mapped_file_source class has the following category of source_tag + direct_tag + closable_tag, and has member functions open(), size(), data(), begin(), and end(). None of these are particularly surprising. After all - a mapped_file has to be open()ed at some point. But under what model as defined in the docs? In other words, how and when is that function called, and how does a stream_buffer know he can call open() on a particular device? Also, the documentation indicates that a source device needs to support the read() function - but mapped_file_source has no read() (maybe because it is convertible to 'direct'?). -- Benjamin A. Collins <ben.collins(a)acm.org> http://bloggoergosum.us

2 1

[iostreams] Patch for latest CodeGear (Borland) compiler
by Nicola Musatti 05 Jan '08

05 Jan '08

Hallo, The patch in http://svn.boost.org/trac/boost/ticket/1294 improves support for the latest version of bcc32, 5.9.2 . If I don't receive objections in two or three days I'll apply it. Cheers, Nicola -- Nicola.Musatti <at> gmail <dot> com Home: http://nicola.musatti.googlepages.com/home Blog: http://wthwdik.wordpress.com/

2 1

Performance of various mutex implementations on Linux
by Phil Endecott 02 Jan '08

02 Jan '08

Dear All, I have been benchmarking different mutex implementations on Linux. The variations that I have considered are the following: NO_LOCKING: - no mutex at all, as a baseline for comparison. SPINLOCK - uses an atomic operation and a tight loop. YIELD - uses an atomic operation like SPINLOCK, and calls sched_yield() if the lock is held by another thread. PTHREADS - uses the pthread_mutex_*() functions; this is what the current Boost mutexes do (as I understand it). The pthread_mutex functions are implemented using the futex() system call. FUTEX - this is my own implementation of the same algorithm that PTHREADS uses internally, based on the futex() system call, but it uses inlining to avoid a function call in the un-contended case. Apart from the PTHREADS method, these should all be considered unproven code; if anyone would like to review them for any subtle concurrency issues or other problems you are very welcome to do so. For background details of futexes, I suggest Ullrich Drepper's paper "Futexes Are Tricky". All of these results were obtained with g++ 4.1.2 with -O3. Data Structure Sizes -------------------- The size of the mutex object, in bytes, is as follows: NO_LOCKING: 0 SPINLOCK: 4 [*] YIELD: 4 [*] FUTEX: 4 [*] PTHREADS: 24 [*] in all of these cases I believe that a single byte, or perhaps even a single bit in the case of SPINLOCK and YIELD, would be sufficient; there may be some obscure alignment issue though. I believe that the large size of the pthreads mutex object is because it supports a number of variants such as recursive and checking mutexes, and timeouts, etc. all in the same structure. My other implementations are all very basic with none of these features. Execution Time, Non-Contended Case ---------------------------------- In many uses mutexes are rarely contended, so optimising the speed of the uncontended locking and unlocking operations is a common objective. I have measured this overhead on three machines: egypt: 1GHz VIA C3 x86 uniprocessor linux-2.6.19 ghana: 266MHz Intel XScale ARM uniprocessor linux-2.6.19 atum: 3.6GHz Intel Pentium 4 x86 2 processor linux-2.6.17 The times in nanoseconds for one lock and one unlock are: egypt ghana atum NO_LOCKING 0 0 0 SPINLOCK 17 30 5.7 YIELD 13 38 6.4 FUTEX 50 95 26 PTHREADS 115 986 39 Execution Time, Contended Case ------------------------------ See my detailed results, linked below, for numbers (the runtimes given are for 10^8 operations). The execution time in the contended case includes both the execution time of the lock and unlock functions and the time spent waiting for the lock to become available; it's not straightforward to distinguish these and I haven't tried to. The broad pattern is that SPINLOCK, naturally, is very slow, and the other methods are relatively fast. In my benchmark, YIELD does better than FUTEX, but this may well be a feature of the structure of my benchmark; I would expect FUTEX to do better with large numbers of threads. PTHREADS gets worse more quickly than other other methods as the number of mutexes and threads increases, perhaps because the larger size of its structures is less cache-friendly. Code Size --------- The code size overhead in bytes for each method is as follows. (Note that this is with -O3, not -Os. My impression with -Os is that it does even less inlining that I expected, e.g. even a static function that's only called once is not inlined. I may investigate this further.) x86 ARM NO_LOCKING 0 0 SPINLOCK 42 32 YIELD 31 56 FUTEX 66 96 PTHREADS 25 28 Observations ------------ I suggest that we seriously consider using an inline futex implementation of mutexes as the default choice on Linux, rather than calling the pthread_mutex functions. This is about twice as fast (on x86) and saves a lot of data size. On the other hand, it increases code size. The code size and speed can be improved further by using a simpler method that just calls sched_yield(). However, this method is less suitable for use as a default implementation because it will have worse complexity in applications with contention: futex() knows which thread to wake up next, while sched_yield() doesn't. ARM Observations ---------------- Most ARM processors have only one atomic instruction, swap. My implementations of SPINLOCK, YIELD and FUTEX make use of this. PTHREADS has abysmal performance on ARM; I suspect, but have not yet checked, that it somehow emulates the atomic instructions that it doesn't have by calling into the kernel. You can find all of the mutex implementation and benchmarking scripts and the full results here: https://svn.chezphil.org/mutex_perf/trunk/ I hope this is of interest. Regards, Phil.

4 4