- Boost - lists.boost.org

[thread] locks.hpp double declaration of unique_lock(unique_lock&& other)
by vicente.botet 21 Jan '09

21 Jan '09

Hi in lines 248-255 of the file boost/thread/locks.hpp we found the unique_lock(unique_lock&& other) declaration twice one withoth explicit and defined and the other with explicit an undefined. #ifdef BOOST_HAS_RVALUE_REFS unique_lock(unique_lock&& other): m(other.m),is_locked(other.is_locked) { other.is_locked=false; other.m=0; } explicit unique_lock(upgrade_lock<Mutex>&& other); Could you say me why this is nesded? _____________________ Vicente Juan Botet Escribá

3 3

[iostreams] Error reporting for buffered streams
by Eric MALENFANT 21 Jan '09

21 Jan '09

By default, filtering_streams do buffering. This means that what is written to them does not necessarily reach the underliyng Filters and Device immediately. It is thus my understanding that, in order to be sure that all data was written, one should somehow flush the buffered data, and then check the stream state before closing it. As a first attempt, I tried flush(), but it does not seem to do what I expected. Looking at the docs, I then found what seemed to be what I was looking for: strict_sync(), whose documentation states that "A return value of true guarantees that all buffered data has been successfully forwarded.". Moreover, the FAQ entry "Can I swap Filters or Devices in the middle of a sequence of i/o operations?" seems to confirm this, as it says: "First call strict_sync. If it returns true, you can safely call set_auto_close(false) and pop one or more components without closing the stream. This applies to instances of filtering_stream, filtering_streambuf and chain." But again, this did not work: strict_sync() returns true although buffered data remains. Consider this contrived example: // A codecvt facet that always fails class always_fail_cvt : public std::codecvt<wchar_t,char,mbstate_t> { protected: virtual result do_out( state_type&, const intern_type*, const intern_type*, const intern_type*&, char* to, char*, char*&) const { return std::codecvt_base::error; } //... }; int main(int argc, char* argv[]) { std::wofstream fs("whatever"); fs.imbue(std::locale(fs.getloc(), new always_fail_cvt)); boost::iostreams::filtering_wostream s(fs); s << L"Hello"; std::cout << std::boolalpha << s.good() << ', '; s.flush(); std::cout << s.good(); return 0; } Running this (on Windows with MSVC 7.1) produces "true, true" The first "true", is not surprising, as the characters are buffered and code conversion has not taken place yet, but the second is. (to me at least). Considering the presence of strict_sync(), I assumed that the "forgiveness" of flush() is intentional, to allow stateful filters to retain buffered data, maybe. However, replacing the "s.flush();" line in the above program with "std::cout << s.strict_sync() << ", ";" does not seem to change things much. Even though a conversion failure occured and data remains buffered, strict_sync() returns true, and the stream's state is still good(). So, to sum up: 1) If my understanding of the semantics of strict_sync() is correct, it seems that indirect_streambuf::strict_sync() should verify that the buffer is empty after the call to sync_impl(), and fail if it is not the case. Something like this maybe: template<typename T, typename Tr, typename Alloc, typename Mode> bool indirect_streambuf<T, Tr, Alloc, Mode>::strict_sync() { try { // sync() is no-throw. sync_impl(); return obj().flush(next_) && ((pptr() - pbase()) == 0); // Added this --^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ } catch (...) { return false; } } 2) Is it "correct" that filtering_stream::flush() does not completely flush without reporting an error? 3) Does this come from an incompatibility between Boost.IOStreams idea of error-reporting (throwing exceptions) and that of the std streambufs (returning EOF, mainly)? In particular, Boost.IOStreams writes in a wrapped std::streambuf by using sputn(), which, AFAI(don't)U, does not have a clear error reporting strategy. IOWs, should write_device_impl<streambuf_tag>::write() be modified so that it reports errors by an exception. Something like this maybe: template<typename T> static std::streamsize write (T& t, const typename char_type_of<T>::type* s, std::streamsize n) // { return t.sputn(s, n); } { std::streamsize count = t.sputn(s, n); if (count == n){ return count; } // If all characters could not be written, check if it is because of an error or only because of blocking if (T::traits_type::eq_int_type(t.sputc(s[count]), T::traits_type::eof())){ throw std::ios_base::failure("write error"); } else{ return count + 1; } } Sorry for the longish post, and thanks for reading up to here, Éric Malenfant

1 0

[random] A much faster Poisson variate generation
by Ross Levine 21 Jan '09

21 Jan '09

Hello, I have modified the poisson_distribution class to use an updated version of Ahrens and Dieter's PD algorithm for generating Poisson variates, as described in "Computer Generation of Poisson Deviates from Modified Normal Distributions." Is this allowed in Boost, and is there interest? The current algorithm is O(mean), while PD is bounded time. In addition, the current implementation fails for mean >> 750. PD does, however, use a lookup table and thus has a non-trivial constructor and needs 36 * sizeof(RealType) + 1 * sizeof(IntType) bytes per poisson_distribution object. Please let me know if this is desired. Ross

2 7

Re: [boost] A PP trick to define a synchronized block java-like macro
by JOAQUIN M. LOPEZ MUÑOZ 21 Jan '09

21 Jan '09

________________________________________ De: boost-bounces(a)lists.boost.org [boost-bounces(a)lists.boost.org] En nombre de Sebastian Redl [sebastian.redl(a)getdesigned.at] Enviado el: martes, 20 de enero de 2009 22:39 Para: boost(a)lists.boost.org Asunto: Re: [boost] A PP trick to define a synchronized block java-like macro > vicente.botet wrote: > > for (VARS DECLARATION, bool __continue=true; > > __continue; > > __continue=false) > > > > > I'm afraid this is not valid syntax. The first clause of the > for-statement is a for-init-statement, which can be either an expression > with a semicolon, or a simple-declaration. A simple-declaration lets you > declare any number of variables, but they must all have the same > declspec, i.e. the same fundamental type (with variations like "pointer > to", "array of" or "function returning"). The following variation would do: for (VARS DECLARATION, *continue_hlp_,**continue_=&continue_hlp_; continue_; continue_=0) Joaquín M López Muñoz Telefónica, Investigación y Desarrollo

5 6

[1.38.0][Spirit] char_class test failing on both trunk and release, all platforms
by Beman Dawes 21 Jan '09

21 Jan '09

The Spirit char_class test is failing on both trunk and release, all platforms. Any progress on fixing this? --Beman

2 1

[1.38.0] [config] Linux release branch link failures
by Beman Dawes 20 Jan '09

20 Jan '09

config_test is failing with link errors on both Linux testers. Any ideas? --Beman "g++" -o "/home/steve/Packages/boost/upstream/testing/release/results/boost/bin.v2/libs/config/test/config_test.test/gcc-4.3.3/debug/config_test" -Wl,--start-group "/home/steve/Packages/boost/upstream/testing/release/results/boost/bin.v2/libs/config/test/config_test.test/gcc-4.3.3/debug/config_test.o" -Wl,-Bstatic -Wl,-Bdynamic -Wl,--end-group -g /home/steve/Packages/boost/upstream/testing/release/results/boost/bin.v2/libs/config/test/config_test.test/gcc-4.3.3/debug/config_test.o: In function `boost_has_pthread_mutexattr_settype::f()': /home/steve/Packages/boost/upstream/testing/release/boost/status/../libs/config/test/boost_has_pthread_ma_st.ipp:21: undefined reference to `pthread_mutexattr_init' /home/steve/Packages/boost/upstream/testing/release/boost/status/../libs/config/test/boost_has_pthread_ma_st.ipp:23: undefined reference to `pthread_mutexattr_settype' /home/steve/Packages/boost/upstream/testing/release/results/boost/bin.v2/libs/config/test/config_test.test/gcc-4.3.3/debug/config_test.o: In function `boost_has_pthreads::test()': /home/steve/Packages/boost/upstream/testing/release/boost/status/../libs/config/test/boost_has_pthreads.ipp:35: undefined reference to `pthread_mutex_trylock' /home/steve/Packages/boost/upstream/testing/release/boost/status/../libs/config/test/boost_has_pthreads.ipp:44: undefined reference to `pthread_create' /home/steve/Packages/boost/upstream/testing/release/boost/status/../libs/config/test/boost_has_pthreads.ipp:52: undefined reference to `pthread_join' /home/steve/Packages/boost/upstream/testing/release/results/boost/bin.v2/libs/config/test/config_test.test/gcc-4.3.3/debug/config_test.o: In function `boost_has_clock_gettime::f()': /home/steve/Packages/boost/upstream/testing/release/boost/status/../libs/config/test/boost_has_clock_gettime.ipp:21: undefined reference to `clock_gettime' collect2: ld returned 1 exit status

3 3

A PP trick to define a synchronized block java-like macro
by vicente.botet 20 Jan '09

20 Jan '09

In Boost.Log we can see how to define a macro that preserv the osream syntax but that logs conditionally. LOG << "Ah" << std::end; We use here the following trick: let the sentence generated by the macro is unterminated and form a correct sentence together with what follows the macro call. #define LOG if (!log::enabled()); else log::os I have recently see in [1] a trick allowing the declaration of a variable local to a block scope but OUT OF THE SCOPE!!!. This will allows to write things like synchronized(mtx_) { // block protected by a scoped lock } or atomic(tr) { // atomic block managed by a transaction } The idea is to place the declaration of the needed variables on the init declaration of a for. So these variables are accessible by the next sentence or block. The condition will let executes its body only once. A variable __continue variable is used to this goal; it initialized to true and set to false on the next part of the for. for (VARS DECLARATION, bool __continue=true; __continue; __continue=false) The definition of the synchronized macro can be #define synchronized(MUTEX) \ for (boost::mutex::scoped_lock __lock(MUTEX), bool __continue=true; \ __continue; \ __continue=false) In order to help when debuging we can define the __lock and _cont variables as macros generating symbols including the line number. #define __concat(a,b) a##b #define _concat(a,b) __concat(a,b) #define __lock _concat(__lock_, __LINE__) #define __continue _concat(__continue _, __LINE__) As the variables are local to the scope, we can use the macro twice on the same line as synchronized(mtx_1) synchronized(mtx_2){ // block protected by a scoped lock on mtx_1 and mtx_2 // ... } See how we can nest the synchronized blocks: synchronized(mtx_1) { // block protected by a scoped lock on mtx_1 // ... 1 synchronized(mtx_2) { // Embeeded block protected by a scoped lock on mtx_2 // ... 2 synchronized(mtx_3) unique_sentence_protected_by_mtx_3(); // ... 2 } // ... 1 } I know that most of the participants to this ML do not like macros too much, even less when we can do without them. { // block protected by a scoped lock on mtx_1 boost::mutex::scoped_lock lock1(mtx_1) ; // ... 1 { // Embeeded block protected by a scoped lock on mtx_2 boost::mutex::scoped_lock lock1(mtx_2) ; // ... 2 { // Artificial Embeeded block protected by a scoped lock on mtx_3 boost::mutex::scoped_lock lock1(mtx_3) ; unique_sentence_protected_by_mtx_3(); } // ... 2 } // ... 1 } But I think that in these cases the language-like macros make the code realy more readable, specially in the case of software transactional memory (STM), see (http://en.wikipedia.org/wiki/Software_transactional_memory) // language-like transaction with retry. try_atomic(t) { transaction_ptr(x) = val; } before_retry {} I wanted simply to share this trick with you, maybe someone wants to use something like that in its own application. Let me know however what do you think. Thanks to the authors of this excelent paper and the DracoSTM library. Hoping this would inspire some of you, Vicente ___________________________________ P.S. the exception based timed locks post http://www.nabble.com/-thread--Exception-based-timed-locks-tp21554444p21554… is also inspired from this article. [1] Toward Simplified Parallel Support in C++ Justin E. Gottschlich, Jeremy G. Siek, Paul J. Rogers and Manish Vachharajani

2 1

filesystem::remove_all error handling
by Zachary Turner 20 Jan '09

20 Jan '09

Has there been any discussion regarding the possibility of filesystem::remove_all allowing a second parameter, a functor or function, to be specified which on any file system error is called with the filename that was attempted to be removed as well as the error code? My application, for example, would like to log every single occurence of failure with a warning, but still delete every possible file that it can, but I see no way to handle this without rolling my own implementation of remove_all. Which is fine if that's what I have to do, but at the same time it doesn't seem that uncommon a case where someone would want to know about every single error, yet still attempt every deletion possible. I guess one tricky aspect of this is that you probably would not want to invoke the error callback if the error code means "directory not empty" and the reason it is not empty is because a file was unable to be deleted below the directory. Side note, but the implementation of remove_all does not appear to be tail recursive either, so even the best compiler will not be able to optimize this into a loop, leading to the potential for a stack overflow in deeply nested directory hierarchies. (Now that I think about it, I'm not even sure what, if any, C++ compilers optimize tail recursive calls). Apologies if either of these two issues has been addressed (I'm using 1.35.0), did not find anything in the archives. Thoughts?

2 1

[Serialization] Improving performance
by Hartmut Kaiser 20 Jan '09

20 Jan '09

Hi all, in one of my projects I have a lot of types (>1000) to be serialized using a pointer to a single base class. At some point we found the serialization/deserialzation time to be O(N*M), where N is the number of types and M the number of classes in the derivation hierarchy. Wondering why this is so significant I started digging and measuring. I found the type information registry used for the void_upcast() to be the culprit. It's a plain std::vector<const void_caster *> ($BOOST_ROOT_1_37/libs/serialization/src/void_cast.cpp:37) which is searched sequentially a lot (once for each derived/base pair for each serialization call). Moreover, this vector isn't even kept sorted. it = std::find_if( s.begin(), s.end(), void_cast_detail::match(& ca) ); ($BOOST_ROOT_1_37/libs/serialization/src/void_cast.cpp:180). Changing this to be a std::set improves the picture significantly! What's the reasoning behind using a std::vector<> instead of a std::set<> or a similar indexed structure? Regards Hartmut

4 3

[thread] Exception based timed locks
by vicente.botet 20 Jan '09

20 Jan '09

Hi, I've read recently about timed locks throwing an exception when there is a timeout. What do you think about this behaviour? Could the thread library provide both? Next follows a piece of code without exceptions while (polling) { t=now()+100; boost::unique_lock<boost::mutex> l1(m1, t); if (l1.has_lock() { boost::unique_lock<boost::mutex> l2(m2, t); if (l2.has_lock() { boost::unique_lock<boost::mutex> l3(m3, t); if (l2.has_lock() { foo(); polling = false; } else execute_on_failed(); } else execute_on_failed(); } else execute_on_failed(); } and now with exceptions while (polling) try { t=now()+100; boost::exception_unique_lock<boost::mutex> l1(m1, t); boost::exception_unique_lock<boost::mutex> l2(m2, t); boost::exception_unique_lock<boost::mutex> l3(m3, t); foo(); polling = false; } catch (timeout_exception& ex) {execute_on_failed(); } What do you think about a try_lock_until, try_lock_for functions while (polling) try { boost::try_lock_for(100, m1, m2, m3); foo(); polling = false; } catch (timeout_exception& ex) {execute_on_failed(); } Best, Vicente

4 6