- Boost - lists.boost.org

[Iostreams] Support std::codecvt<char> and std::codecvt_base::noconv in boost::iostreams::code_converter
by Ferdinand Prantl 09 Jul '08

09 Jul '08

Hello, I found two problems when using boost::iostreams::code_converter with char as source (internal) type for the conversion. Firstly, I was not able to use facets parameterized with char extracted from std::locale and secondly, my test run in an endless loop if the facet returned std::codecvt_base::noconv from do_in() or do_out(). I would like to use the boost::iostreams::code_converter in a class parametrized either for char or wchar_t using the default std::codecvt. This code works for wchar_t: namespace io = boost::iostreams; class test { io::stream<io::code_converter<xxxodevice, io::default_codecvt> > output; ... }; However I cannot make in templated like this without inheriting from std::codecvt and pasing the inherited type to io::code_converter instead of std::codecvt: template <class C> class test { io::stream<io::code_converter<xxxodevice, std::codecvt<C, char, std::mbstate_t> > output; ... }; The compiler complains about the usage of std::codecvt: error C2248: 'std::codecvt<_Elem,_Byte,_Statype>::~codecvt' : cannot access protected member declared in class 'std::codecvt<_Elem,_Byte,_Statype>' io::detail::codecvt_holder<Codecvt> in codecvt_holder.hpp stores a copy of Codecvt but std::codecvt defines only protected destructor: template<typename Codecvt> struct codecvt_holder { ... Codecvt codecvt_; }; Standard std::codecvt specialized for wchar_t is solved by its explicit specialization storing the pointer: struct default_codecvt { typedef wchar_t intern_type, from_type; typedef char extern_type, to_type; typedef std::mbstate_t state_type; }; template<> struct codecvt_holder<default_codecvt> { ... const codecvt_type* codecvt_; }; Obviously it supports the most usual usage with an instance extracted from std::locale which owns it: typedef std::codecvt<wchar_t, char, std::mbstate_t> codecvt_type; ... = &std::use_facet<codecvt_type>(locale); What about facets not converting from sequence of wchat_ts but chars? If I imbue such facet to a locale I am not able to extract it and use it in io::code_converter: template <class C> class test { io::stream<io::code_converter<xxxodevice, std::codecvt<C, char, std::mbstate_t> > output; void imbue(std::locale const & locale) { output.imbue(locale); } }; I helped it by converting io::detail::default_codecvt and the specialization of io::detail::codecvt_holder for it in codecvt_holder.hpp to this: template <class C> struct basic_default_codecvt { typedef C intern_type, from_type; ... }; template <class C> struct codecvt_holder<basic_default_codecvt<C> > { typedef std::codecvt<C, char, std::mbstate_t> codecvt_type; ... }; typedef basic_default_codecvt<wchar_t> default_codecvt; Now I was able to compile my slightly modified code: template <class C> class test { io::stream<io::code_converter<xxxodevice, io::detail::basic_default_codecvt<C> > output; void imbue(std::locale const & locale) { output.imbue(locale); } }; After being able to compile it a simple output code run endlessly. I found out that the facet in the default locale did not convert but it did not return true from always_noconv() from some reason. io::code_converter can cope with it - it copies the input buffer to the output one but it does not increase the pointer to the next character in the output buffer nor does it decrease the total bytes processed in io::code_converter::write. I am not sure about the behavior of the coverter if the facet returns std::codecvt_base::noconv. Currently it copies the buffer if internal type is the same as external one (both char) otherwise it does nothing (?). Should it not copy or assert if they are diferent? At least the writing should succeed if there was no conversion done and the characters were copied. I tried to modify the code accordingly and it succeeded then. I am attaching a patch to code_converter.hpp and codecvt_holder.hpp from boost 1.35.0. I could provide a test for it too. Thank you, Ferda

1 0

[Iostreams] What about making code_converter from boost::iostreams flushable?
by Ferdinand Prantl 09 Jul '08

09 Jul '08

Hello, I would like to wrap an output stream buffer (std::streambuf) with boost::iostreams::code_converter to be able to use it both raw and with ont-the-fly encoding by the boost::iostreams::default_facet. namespace io = boost::iostreams; xxxstream stream; stream << "raw"; io::stream<io::code_converter<xxxodevice> > wrapper(*stream.rdbuf()); wrapper << L" and converted"; stream << " and another raw"; Running the code writes "raw and another raw and converted" instead of "raw and converted and another raw" to the stream. I found that first in destructor of io::code_converter is the conversion buffer flushed causing the output. I could create a new instance of io::code_converter for every output and always close it: namespace io = boost::iostreams; xxxstream stream; stream << "raw"; { io::stream<io::code_converter<xxxodevice> > wrapper(*stream.rdbuf()); wrapper << L" and converted"; } stream << " and another raw"; But I would feel better being able to reuse the io::code_converter. (Actually, I have it as a private field in a class now.) I would like to make the io::code_converter flushable to be able to combine the output more conveniently: namespace io = boost::iostreams; xxxstream stream; stream << "raw"; io::stream<io::code_converter<xxxodevice> > wrapper(*stream.rdbuf()); wrapper << L" and converted" << std::flush; stream << " and another raw"; I added the support to io::code_converter rather that inheriting from it. What do you think about it? Would you accept it to boost or is it better to make flushable my inherited class? I am attaching a patch to code_converter.hpp from boost 1.35.0. I could provide a test for it too. Thank you, Ferda

1 0

[Iostreams] code_converter from boost::iostreams fails with bidirection device
by Ferdinand Prantl 09 Jul '08

09 Jul '08

Hello, I would like to wrap an input/output stream buffer (std::streambuf) with boost::iostreams::code_converter to be able to use it both raw and with ont-the-fly encoding by the boost::iostreams::default_facet. namespace io = boost::iostreams; xxxstream stream; stream << "raw "; io::stream<io::code_converter<xxxiodevice> > wrapper(*stream.rdbuf()); wrapper << L"and converted"; I created a device wrapping std::streambuf. Pure input device (using io::source_tag) and pure output device (using io::sink_tag) were working. Combined device (using io::bidirectional_device_tag) failed with access violation in destructor (flushing and closing) of the io::code_converter in code_converter.hpp: template<typename Device, typename Codevt, typename Alloc> std::streamsize code_converter<Device, Codevt, Alloc>::write (const char_type* s, std::streamsize n) { buffer_type& buf = out(); ... std::codecvt_base::result result = // buf.eptr() was null here cvt().out( buf.state(), s + total, s + n, nint, buf.eptr(), buf.end(), next ); ... } I wondered how it is possible that the output buffer was not initialized. I learnt that the buffer was intentionally skipped in code_converter.hpp: template<typename Device, typename Codecvt, typename Alloc> struct code_converter_impl { ... void open(const Device& dev, int buffer_size) { ... // following condition is false for bidirectional if (can_write::value && !is_double::value) { buf_.second().resize(buffer_size); buf_.second().set(0, 0); } ... } ... }; The reason is the definition of a bidirectional device in categories.hpp: struct bidirectional : virtual input, virtual output, detail::two_sequence {}; It is detected in the io::code_converter turning off the initialization: typedef is_convertible<device_category, two_sequence> is_double; It means that I cannot use io::code_converter with io::bidirectional combining both io::input and io::output tags. Unfortunately I was not able to use these both explicitely: // original: typedef io::bidirectional_device_tag category; struct category : io::device_tag, io::input, io::output {}; The compiler complained about it with a message I was not able to fix: error C2784: 'std::basic_ostream<_Elem,_Traits> &boost::operator <<( std::basic_ostream<_Elem,_Traits> &,const boost::shared_ptr<Y> &)' : could not deduce template argument for 'std::basic_ostream< _Elem,_Traits> &' from 'boost::iostreams::stream<Device>' So far, I found just one working solution without changing boost sources - using two objects; one for reading, another one for writing: xxxstream stream; stream << "raw "; io::stream<io::code_converter<xxxidevice> > inwrapper(*stream.rdbuf()); io::stream<io::code_converter<xxxodevice> > outwrapper(*stream.rdbuf()); outwrapper << L"and converted"; However, I would like to use one object if it was possible. In my case, all writing is always finished before reading and vice versa. Encoding states in std::codecvt would not overlap. What would you suggest as a solution for input/output devices in io::code_converter? 1. Stick with the two objects - one for input, one for output. 2. Fix io::code_converter to accept devices with io::bidirectional tag. Either initialize both buffers or reuse the input one. 3. Use different tag than io::bidirectional. Which one? 4. Maybe I am wrapping std::streambuf in a wrong way. Is there another? I am attaching a simple example of a stripped source demonstrating the issue on std::stringstream. Thank you, Ferda

1 0

[Boost.Log1]: Bug in boost/log/detail/ts_posix.hpp
by Jens Seidel 09 Jul '08

09 Jul '08

Hi, I know that version 1 of John's logging library was never accepted and is probably not wildly used anymore nevertheless I found a nasty bug in it. It's strange but even after applying valgrind I found initially not the reason (probably I just assumed John's code is OK and it's something else) and it took some time until I resolved it. OK, for all users of this lib here is the patch: boost/log/detail/ts_posix.hpp: @@ -51,7 +51,7 @@ mutex & operator = ( const mutex & Not_Implemented); mutex( const mutex & From); public: - mutex() : m_mutex() { + mutex() : m_mutex(), m_count(0) { pthread_mutexattr_t attr; int res = pthread_mutexattr_init(&attr); assert(res == 0); I haven't checked whether this occurs in version 2 as well. Jens

3 3

Fusion Question
by Tom Brinkman 09 Jul '08

09 Jul '08

Heres a link to a simple unit test that demonstrates the layout library. http://www.reportbase.com/unit_test.hello.html

2 1

Re: [boost] How do you make MPL-compatible template meta-functions?
by joaquin＠tid.es 09 Jul '08

09 Jul '08

Daryle Walker escribió: > On Jul 9, 2008, at 3:02 AM, joaquin(a)tid.es wrote: > [...] >> You can use boost::mpl::equal to test this: >> >> BOOST_STATIC_ASSERT >> ( (boost::mpl::equal<distinct_integral_bit_counts, >> boost::mpl::list_c<int, 8, 16, 32> >::value) ); >> } >> >> Other than that, your type_to_digit_count is perfectly lambda- >> cabable AFAICS. HTH, >> > > Thanks. I switched to mpl::equal and it compiled. I later switched > to BOOST_MPL_ASSERT. (Remember to take _out_ the "::value" part!) I > originally had the "integral_c" part of "type_to_digit_count" as a > member called "type," instead of as a base class, then I switched > back and forth. Both ways work. Why is that? Because integral_c (and all other integral constant types in Boost.MPL) evaluates to itself, that is, it has the form: template<typename T, T N>struct integral_c { typedef integral_c<T,N> type; ... }; > And which way is best > (if they're not equally good, and without a better third alternative)? > I don't think it makes any difference in compile times or otherwise, but deriving from integral_c is terser and more MPL-idiomatic. Joaquín M López Muñoz Telefónica, Investigación y Desarrollo

1 0

How do you make MPL-compatible template meta-functions?
by Daryle Walker 09 Jul '08

09 Jul '08

Here's what I'm trying (to redo integer_test.cpp): //============================== #include <boost/config.hpp> #include <boost/limits.hpp> #include <boost/integral_c.hpp> #include <boost/mpl/list.hpp> #include <boost/mpl/transform.hpp> #include <boost/static_assert.hpp> #include <boost/type_traits.hpp> #include <climits> namespace { // List all the conventional integral types, without the ones that are like // a strong-typedef of a lower type typedef boost::mpl::list<unsigned char #if USHRT_MAX > UCHAR_MAX , unsigned short #endif #if UINT_MAX > USHRT_MAX , unsigned int #endif #if ULONG_MAX > UINT_MAX , unsigned long #endif #if defined(BOOST_HAS_LONG_LONG) && ((ULLONG_MAX > ULONG_MAX) || (ULONGLONG_MAX > ULONG_MAX)) , boost::ulong_long_type #elif defined(BOOST_HAS_MS_INT64) && (0xFFFFFFFFFFFFFFFFui64 > ULONG_MAX) , unsigned __int64 #endif > distinct_unsigned_types; typedef boost::mpl::transform<distinct_unsigned_types, boost::make_signed<boost::mpl::_1> >::type distinct_signed_types; // New type list: integral type -> (wrapped) number of bits template < typename T > struct type_to_digit_count : boost::mpl::integral_c<int, std::numeric_limits<T>::digits> { }; typedef boost::mpl::transform<distinct_unsigned_types, type_to_digit_count<boost::mpl::_1> >::type distinct_integral_bit_counts; BOOST_STATIC_ASSERT ( (boost::is_same<distinct_integral_bit_counts, boost::mpl::list_c<int, 8, 16, 32, 64> >) ); } //============================== I originally did not have that last line there with the "BOOST_STATIC_ASSERT," but I put it in after some errors when doing a compiler-sanity check while coding. I cleaned up the other errors, but the compiler complains that the first argument of "is_same," i.e. "distinct_integral_bit_counts," is an invalid template argument. I don't think my definition of "type_to_digit_count" is lambda-safe, like "make_signed" is. How do I get it that way? (Or is the problem in the definition of "distinct_integral_bit_counts"?) -- Daryle Walker Mac, Internet, and Video Game Junkie darylew AT hotmail DOT com

1 0

[test][gcc] return value support broken with "-pedantic"
by Juergen Hunold 09 Jul '08

09 Jul '08

Hi ! The latest changes to Boost.Test broke compilation with gcc in "pedantic" mode. This seems to be: http://svn.boost.org/trac/boost/changeset/47153 Just add "<toolset>gcc:<cxxflags>-pedantic" to libs/test/test/Jamfile.v2 and recompile. This will bail out with: gcc.compile.c++ /build/bjam/boost_1.36.0/libs/test/test/output_test_stream_test.test/gcc-4.2/debug/output_test_stream_test.o output_test_stream_test.cpp: In member function 'void test_constructor::test_method()': output_test_stream_test.cpp:30: error: ISO C++ forbids braced-groups within expressions I've disabled "return value support" locally (by commenting out test_tools.hpp line 86) but would like to have a conformant solution. Should we rise the warning level by _always_ activating "-pedantic" at least for the test suite ? Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! Ingenieurgesellschaft für * voice: ++49 511 262926 57 ! Verkehrs- und Eisenbahnwesen mbH * fax : ++49 511 262926 99 ! Lister Straße 15 * juergen.hunold(a)ivembh.de ! www.ivembh.de * * Geschäftsführer: ! Sitz des Unternehmens: Hannover * Prof. Dr.-Ing. Thomas Siefer ! Amtsgericht Hannover, HRB 56965 * PD Dr.-Ing. Alfons Radtke !

4 6

[test] Serious breakage - please fix or revert immediately
by Beman Dawes 09 Jul '08

09 Jul '08

Yesterday's changes to Boost.Tests appear to be breaking many libraries on gcc compiles. Please fix or revert immediately. --Beman Examples: ../boost/test/impl/test_main.ipp(40): error: destructible entities are not allowed inside of a statement expression BOOST_CHECK( test_main_result == 0 || test_main_result == boost::exit_success ); In file included from ../boost/test/minimal.hpp:37, from ../libs/crc/test/crc_test.cpp:23: ../boost/test/impl/execution_monitor.ipp:60:1: error: unterminated #if ../boost/test/impl/execution_monitor.ipp:24:1: error: unterminated #ifndef

2 2

Fusion Question
by Tom Brinkman 09 Jul '08

09 Jul '08

>> Have you looked into Adobe's Eve library, described in >> http://lambda-the-ultimate.org/node/563 Oh, yes. definately; and if its as good as Adobe's GIL library, I'm sure it would be a most welcome addition to boost. An Adobe sponsered layout engine would be very welcome, I'm sure. As you know, I use GIL extensively in my own graphics layout library. I think that it represents many of the best-practices that can be found on boost. As far as my own boost::layout library, I'd be happy to merge it with adobe's or just add extensions to it. I'll let you and others guide me on that.

3 3