Re: [boost] binary_iostreams library

From: Jerry Schwarz (jerry_at_[hidden])
Date: 2007-05-31 12:56:04

I agree with you that the current streambuf is a mess. If you want to  
create a new binary_streambuf and build your library on that I  
wouldn't object.  But you didn't do that, you built a library without  
any other transport mechanism. At heart my suggestion is to have  
binary_streams built based on some transport mechanism, and have that  
mechanism (whatever it is) highly visible.

On May 31, 2007, at 2:09 AM, Sebastian Redl wrote:

> Jerry Schwarz wrote:
>> For what its worth,  I always envisioned alternative top level IO
>> classes  using  streambuf's as the underlying transport. The central
>> class of the iostream library is really streambuf and I've long
>> regretted not calling the library "streambuf".  I always encourage
>> people to use streambuf's directly when all they want to do is
>> transport bytes.
>>
>> So I like the idea of this library.  It is more consistent with my
>> ideas of how "iostream" should be used than is using istream or
>> ostream directly to read and write binary files.   But I would prefer
>> it if binary_istream, binary_ostream and binary_iostream had a
>> constructor that took a streambuf argument and that (at least
>> optionall) ownership would be transfered to the streambuf.  That is
>> the stream would be responsible for freeing it.
>>
> The C++ streambuf class is still character-oriented, not byte- 
> oriented.
> In particular, it needs a char_traits parameter (and the associated
> state_type typedef and length operation, both of which are meaningless
> for bytes), the importance of which I question anyway. What's worse,
> it's the stream buffer that's responsible for converting between the
> "internal character representation" and the "external character
> representation". This means that, unless it's in binary mode (and
> perhaps even then, partially), the stream buffer does character set
> conversion (if it's a wide stream, or perhaps some exotic system that
> enforces a specific encoding for text files) and newline conversion.
> Another example of the character-orientation of the buffers is that
> there is a stringstream, not a vectorstream.
>
> Having binary streams that can work with classical stream buffers is
> good for retaining existing work, but I consider it unacceptable as  
> the
> mandatory underlying transport.
>
>
> I think a new I/O system should be built in layers like this:
>
> lowest) The byte transport. It always seemed weird and a severe
> shortcoming to me that C++ basically assumed the base for external  
> data
> to be text. The basic underlying unit of external data should be the
> byte (i.e. unsigned char in the C++ type system). To represent this
> level, I think a source/sink/filter stack system like Boost.IOStreams
> would be appropriate, but oriented strictly towards binary. Buffers  
> are
> on this level.
>
> --) On top of the byte transport, there is the binary I/O layer,  
> akin to
> the proposed binary_iostream here. It handles the fun part of binary
> I/O, such as assembling bytes into multibyte types (endianness) or
> rearranging the internal data within bytes (also endianness, on  
> systems
> where char is large), and perhaps even converting to and from a
> canonical external float representation.
>
> --) Aside: Binary serialization of objects could build on the  
> binary I/O
> layer.
>
> --) On top of the binary I/O comes the text conversion layer. This
> should be a comprehensive character set and encoding library,  
> capable of
> converting between the internal character set(s) and external
> representations. The external encoding should be chosen by compiler
> default, locale or explicitly passed name. This builds on the  
> binary I/O
> and not directly the byte transport because the base type of a  
> character
> representation can be a multi-byte entity, e.g. in the UTF-16 encoding
> with 8-bit bytes.
>
> --) At this point, another filter layer could be inserted, to have
> filters that work on the text level. This layer could be made
> responsible for handling line ending conversion.
>
> highest) Finally, once we've got text, we add a formatting layer like
> the current iostreams.
>
> This gives each layer a clearly defined function, and the programmer a
> great choice between ease of use and flexibility.
>
> While we're at it, the interface should consider the possibility of
> non-blocking and/or async I/O.
>
> Sebastian Redl
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/ 
> listinfo.cgi/boost