- Boost - lists.boost.org

Empty member optimization
by Joao Abecasis 10 Sep '04

10 Sep '04

Hi! I have written a small utility class to provide "empty member optimization" for arbitrary types. The implementation makes use of the empty base class optimization (EBCO) but avoids injection of the class. A uniform and contained interface is exposed wether the optimization applies or not: template <typename T, size_t Index> compressed_pair { // several constructors may be provided but, for now, // default and copy constructor typedef T value_type; value_type & get(); value_type const & get() const; }; I believe this could be used as the basis for a compressed_tuple. Here's a sample use: template <typename T0, typename T1> struct A : private compressed_member<T0> , private compressed_member<T1, 1> { typedef compressed_member<T0> member0; typedef compressed_member<T1, 1> member1; T0 const & first() { return base0::get(); } T1 const & second() { return base1::get(); } }; There is a catch with the current implementation: some regular empty member (or regular empty member of some compressed_member) could end up sharing address with an empty compressed_member. This happens because compressed_member<T> does not inherit from T but hides this "relationship" from the compiler. Is there any interest in such an utility? To test compressed member I wrote another (toy) utility: empty_store. empty_store is an empty class with a static map that associates an instance's address to data. Checks are made everywhere to ensure that no two instances share the same address. The code can be found (zipped) here: *http://tinyurl.com/4e85l* Best regards, João Abecasis

4 8

ublas indirect_array, shared_array, array_adaptor doc?
by Neal D. Becker 10 Sep '04

10 Sep '04

I noticed in ublas/storage.hpp a number of interesting-looking types, such as array_adaptor, shared_array, indirect_array. I did not find any documentation on these.

1 0

class random_device from <boost/nondet_random.hpp> has no implementation
by steven.robbins＠videotron.ca 10 Sep '04

10 Sep '04

Hi, Most of the boost libraries' implementation is either in header files, or in a compiled library. Class boost::random_device is an exception: the implementation is found in libs/random/random_device.cpp, but this file is not built into any library. This issue has been raised by a Debian user (http://bugs.debian.org/261719) and earlier (http://lists.boost.org/MailArchives/boost-users/msg03506.php) but never addressed. I wanted to confirm whether this is a bug or is by design. Thanks, -Steve

1 0

Re: [boost] IOStreams Formal review -- Guide for Reviewers
by Jonathan Graehl 09 Sep '04

09 Sep '04

First, let me apologize for not being able to review the actual code (yet). The interface and correctness/performance of implementation are all I really care about for now :) >There are some special cases where the copying is wasteful, though. For >instance: > >1. If filter1 is just a passive observer, simply counting the number of >occurences of '\n', or copying the data to a logging stream, then the end user >should really be writing directly to buf2. Filter1 could process the data using >the same buffer as filter2. > >2. Same as above, but filter1 also modifies the data in-place, making >character-by-character modifications. (E.g., a toupper filter). This can be >handled the same way. > >3. If 'resource' is a stream or stream buffer, it could be assumed to do its own >buffering. In that case, filter2 should be writing directly to resource, instead >of to streambuf3: > > filter2.write(resource, buf2, buf2+ n2). > >These three cases can be handled easily by modifying the existing framework. I >didn't add special treatment because it occurred to me rather late in >development. > > > I'd certainly feel a bit more proud of the library if it handled these cases (1 and 3 seem most important). It seems well worth a few days' delay. >There is another class of cases in which the current setup is wasteful, but I >think it is rather domain-specific: > >4. Most filters in the chain modify only small parts of character sequences, >leaving big chunks unchanged. > > > Well, basic_newline_filter would do this - at least when replacing CRLF with a single '\n' character. >To optimize case 4 would require a major library extension. My feeling is that >it is not necessary at this point, but I'd like to know what others think. > > > It should certainly wait if you don't have a design already in mind. The library is good enough without this. >----------------------------------------------------- > >Part III: Interface questions: > >1. How to handle read and write requests which return fewer characters than >requested, though there has been no error, and EOF has not been reached. I think >some answer to this question is necessary to allow the library to be extended >later to handle models orther than ordinary blocking i/o. I mention three >possibilties here, http://tinyurl.com/6r8p2, but only two are realistic. I'm >interested to know how important pople think this issues is, and what is the >best way to resolve it. > > > I think #2 would be most in line with what people are used to under Posix (-1/EAGAIN). Blocking (option 1) actually doesn't make any sense at all except at the ends (source/sink), unless you put each filter in a chain into its own thread and use something like semaphores. I suppose the idea is, if you're in the middle of a filter chain, and somebody gives you some input which would overflow your buffer, you attempt to empty out your buffer to the next guy, but if he can't take enough of it to allow you to accept the whole input (or even, none of it), you have to tell the guy who sent it to you you can't take it, and he has to hold onto it in his buffer. That seems reasonable. I might in fact prefer that my source or sink resource act like #1 (block until at least one character or EOF), and I assume this would be the default behavior if I open it in the default, blocking mode, but it isn't possible to have filters act that way; they need to pass the "can't take your data" feedback all the way back through the stack to the end user, who then needs to hold onto it and select/spin/whatever on the underlying sink resource ... >2. The stack interface. Is the interface to the underlying filter chains rich >enoguh? Originally is was similar to std::list, so that you could disconnect >chains at arbitrary points, store them, and reattach them later. I decided there >wasn't much use for this, so I simplified the interface. > > I'm sure someone, somewhere, sometime will want to perform splices/appends on filter chains, but I can't imagine why, either. At least, keep the simple interface and put the more complicated, flexible one in the appendix. >3. Exceptions. James Kanze has argued repeatedly that protected stream buffer >functions should not throw exceptions (http://tinyurl.com/5o34x) I try to make >the case for exceptions here: http://tinyurl.com/6r8p2. What do people think? > > > > I sympathize with both arguments; either way seems fine to me. There is no real performance penalty for an exception that is thrown at most once per stream (EOF), but he's right that the existing interface (which end users never see) seems to specify a return value in that case. But, as you say, if you want to support async IO, it's moot - you do have to return the number of characters successfully read/written, so you need the std::streamsize return type, so ... you may as well return EOF instead of throwing it. That is, I see no reason to throw an EOF value if you support async IO (which I think would be lovely). >So the question is: Should an open() function be added to the closable >interface, to eliminate the need for first-time switches? Alternatively, should >there be a separate Openable concept? > > > Without a doubt, an Openable concept. If you add open() to Closeable you'd really want to change the concept name ;) For example, if I implement a first-time flag (no real hardship), I'll have to remember to add the first-time test not only when data is processed, but also when the stream is closed (need to handle empty input properly). I suspect people will forget this at least once. There's also a minor performance gain: the interface would be called when the stream is initialized, I assume, and not require a first-time flag check with each use. Admittedly, inconsequential. -Jonathan Graehl

2 6

[boost.range] comparison operators for iterator_range and sub_range
by Thorsten Ottosen 09 Sep '04

09 Sep '04

Dear all, Another small issue have surfaced regarding operator==(), operator!=() and operator<() (the latter currently not there). The current behavior is compare the two underlying iterators. I would suggest to change this to compare the objects in the underlying range. The motivation would be the follwoing: 1. comparing the iterators is seldom useful and can be done explicitlty or by a new member function equal() 2. comparing the objects is the range is highly useful and allows for code like sub_range<string> sub = ...; if( sub == "foo" ) 3. operator<() cannot be defined for iterators in general and hence we couldn't put iterator_range into a set/map; Any comments? An additional question would be if operator> operator>= etc really needs to be defined? I would support just to have ==, !=, < to start with. br Thorsten

3 4

Incorrect XML output using boost::serialization
by Brian Riis 09 Sep '04

09 Sep '04

Using boost CVS (boost-04-09-05-2200), I am using the serialization library (which generally is a real life-saver, btw. Thanks Robert!!!) Anyway, I have been using the XML output for a few days, and I just had a good look at it, and realized it is not well-formed. > <?xml version="1.0" encoding="UTF-8" standalone="yes" ?> > <!DOCTYPE boost_serialization> > <boost_serialization signature="serialization::archive" version="3"> > <m_db class_id="0" tracking_level="0"> > ... The boost_serialization STag above is never matched by a ETag, nor is it an EmptyElem apparently. Is this intended? Or did I miss something while writing the serialization code? I.e. am I supposed to close it myself, or something? It is a minor thing, I know, but it _will_ make any standards-conforming XML-viewer reject it. Other than that, as I mentioned above, Boost.Serialization has saved my life recently (rewriting to use it was by far easier than debugging the I/O routines I was using, and they were somehow _seriously_ messed up). Great work! -- /Brian Riis

1 0

Re:[boost] Re: IOStreams formal review start
by Robert Ramey 09 Sep '04

09 Sep '04

>I decided not to use iterators because I concluded that it would miss the >opportunity for many important optimizations that can be made when one is >presented with a contiguous buffer full of characters instead of one >character at a time. For example, I don't think zlib is very efficient if >you pass it one character at a time. >Still ... >I wish I had looked at the Dataflow Iterators in your library when I first >read your message! I put it on my mental 'to do' list then promptly forgot. >It looks like Dataflow Iterators and input/output filters are intended to >solve exactly the same sort of problems in many cases. So there are these >obvious questions: I didn't want to make a big issue of this because it amounts to comparing what currently the sketch of an idea (my dataflow iterators) to a fully developed and implemented idea (your iostreams). It's an unfair (to you) and often misleading type of comparison. So I see your library getting accepted more or less as it is because it works, well documented and addresses a real and pressing need. Dataflow iterators really became practical with Dave's new iterator façade/adaptor library - with its very formal definition. But, in spite of the fact that they have been used successfully in the serialization package for doing some of the things your library does, Dataflow iterators are a little too fragile for general use. Someday you or someone might want to re-implement some of your library with dataflow iterators - or not. At that point it would become an implementation issue. I'm very pleased that someone has seen the appeal of this idea. I sort of feel that I'm out there by myself with my ideas. I think its because I'm getting old. Anyway. >1. Where the functionality overlaps, which approach is faster? One of the prime motivations for the dataflow iterators is gain absolutely maximum speed. This should occur by collapsing of composed inline functions by C++ optimizing compilers. My limited investigation into this convinces me that this occurs as I would hope. >2. What can your approach do that mine can't? >3. What can my approach do that yours can't? Dataflow iterators is basically a compile-time composition concept while your method is basically a run-time composition concept. There in lies the difference Compile time composition a) compilers can collapse inline code for maximum speed b) each composition of filters is recompiled - can imply replicated almost the same code. Code bloat possible c) new chains can't be made at runtime under program control - its too late to recompile. Runtime composition a) filter chain can be built as needed under program control. b) each filter is compiled only once and reused. No code bloat c) no opportunity for "inter-filter" optimization - the code is compiled before the filters are composed. >4. If your approach is better all around, or in a broad class of cases, can >I adapt my library to use it? (I think so -- the pipe notation, which you >don't like, can give the compiler a clue which filters should be fused at >compile time.) My main complaint with the pipe notation is that, is doesn't permit the following idea to be expressed without destroying its original elegance: Encrypt( Cat( Stream 1, Stream 2 ) ) That is, that you have to break it (the pipe idea) in order to save it. Also I believe that implementing this at compile time by overloading operators would be a lot of work for mere syntactic sugar. In fact this hides the operation of your chaining scheme rather than keeping it transparent. This is the same reason I like to see the streambuf/stream clarified rather than hidden. Hiding is often a good idea - unix command line pipes - but I don't think its a great idea for programmer libraries. I want sharp tools whose application and usage are obvious so that I can compose them into what I need. I don't want a wrapped up solution. So I like a smaller library (e.g. streambuf) with examples (for cut and paste) on how to use it. I also think it's easier to document. I don't think you mess with your library in a major way. Separating out streams/buffers is useful conceptually but that's really mostly a matter of documentation. Basically you should finish what you started, let people start using it then at your leisure if you have the inclination look in this idea. 5. If my approach is better, can your library use it? (BTW, How do you use the Dataflow Iterators in your library?) My use case for this functionality is the following I want a zlib compression filter and an encryption filter. I want to be able to use them in a stream either as compress | encrypt or encrypt | compress. Your library can do that. So the problem is solved in the short term. Its interesting how my thinking has evolved on this. First I thought what I need was a stream adaptors to adapt a stream using filters. After I became more familiear with streams and streambufs I realized that this functionality didn't belong in streams but in streambufs - exactly where you put it. I was much encouraged by your efforts and approach - and Jeff Garland's previous accomplishment of adding zlib compression to a streambuf. Alas, I've now come to understand where codecvt facet fits in here. My current thinking - its just speculation at this point - is composition of codecvt facets using dataflow iterators. This would result in: a) a codecvt_facade - template for making a codecvt class given a dataflow iterator chain at compile time. b) dataflow iterators for encrypt, compress, base64, wchar_t to mbchar, mbchar to wchar_t, wchar_t to utf-8 etc. c) a method - codecvt_adaptor ? - for composition of arbitray codecvt facets. This would permit us to use other existing pre-compiled facets like translating to Chinese mult-byte characterset. So given the above, we could make a custom codecvt facet which would take the output, compress it, encrypt it, convert to base64, convert to Chinese character set with one compile time statement and use that facet with any stream. Except for the Chinese part which used a pre-compiled element, the compiler would optimize away all redundant copying. Naturally this codecvt facet on steroids could be used with any streambuf without alteration. (or even recompilation). So the "final division of effort would be" Streams - rendering types like int, float, etc as char or wchar strings. Streambuf - buffering Codecvt_facet - filtering/transformation Bear in mind that this glosses over details like the fact that there's a lot of annoying little differences in standard libraries in codecvt_facet interface, function prototypes, etc. GCC used 4 by wchar_t, window uses 2 byte wchar_t, endien issues, on and on and on. I've had problems with several libraries that I don't believe handle codecvt facets properly. Just using the utf-8 facet written by Ron Garcia - (nice job - no errors) has been a trial due to these issues. >I'm going to try to re-implement some of your iterators as Filters, and see >what happens. Feel free to experiment at your leisure. But don't get distracted. Starting something is easy - finishing something is very hard. You're in a position to get something of substance actually finished and in the hands of real users. Don't lose focus on that. In the serialization library - composition of dataflow iterators is used to implement the following: Escaping/unescaping html text Binary <-> base64 translation Wstring <-> string translation Maybe others - I don't remember The dataflow concept was once and object of research interest as a fundamental programming paradigm. It's somewhat to related to functional programming in this regard. My dataflow iterator implementation is somewhat similar to the FP package reviewed earlier this year in that it implements a different programming paradigm using the facilities of C++ templates. However, its much less ambitious and much more focused on immediate C++ problems rather producing a complete implementation of another programming paradigm. One thing is clear to me. We're going to see lots of new things in the future of programming. Much more than people think. Robert Ramey

2 1

Re: typeof
by Arkadiy Vertleyb 09 Sep '04

09 Sep '04

Hi Goran, > First, why there isn't a way to register wstring type, even though I've registered wchar_t? I've > managed to use it only after I've registered basic_string template with one template parameter. > Seems inpractical... There are a few choices here: 1) BOOST_TYPEOF_REGISTER_TYPE(wstring); 2) BOOST_TYPEOF_REGISTER_TYPE(wchar_t); BOOST_TYPEOF_REGISTER_TEMPLATE(std::basic_string, 1); 3) BOOST_TYPEOF_REGISTER_TYPE(wchar_t); BOOST_TYPEOF_REGISTER_TEMPLATE(std::basic_string, 3); BOOST_TYPEOF_REGISTER_TEMPLATE(std::char_traits, 1); BOOST_TYPEOF_REGISTER_TEMPLATE(std::allocator, 1); Pre-registration of STL types would be a very nice addition to the library. For now I am avoiding this, and trying to focus on the facility itself. > Second problem, no matter whether I've used wstring or basic_string for registration, I wasn't able > to get a type of wstring::iterator expression! This is much more complicated problem, and I am not sure it can be resolved... The problem is that you can't specialize (or overload) on a nested types. The Standard clearly says that this is a non-deducible context. So we can't do something like: template<class V, class T> encode_type<V, T::iterator>{}; // compiler error! This is VERY unfortunate, and I 've been trying to think of workarounds, but so far I only came up with one that is quite limited and ugly... So I decided to put this on hold. Regards, Arkadiy

2 2

Test of list access through gmane -- please ignore
by Larry Meehan 09 Sep '04

09 Sep '04

Test of boost list access through gmane -- please ignore.

1 0

Re: [boost] Re: Empty member optimization
by JOAQUIN LOPEZ MU?Z 09 Sep '04

09 Sep '04

----- Mensaje original ----- De: Jonathan Turkanis <technews(a)kangaroologic.com> Fecha: Jueves, Septiembre 9, 2004 6:44 pm Asunto: [boost] Re: Empty member optimization [...] > Second, I don't have time to look into the details of your > implementation, but I > wan't to make sure you're aware of the EBO problems Dave Held ran > into with his > policy_ptr, based on the loki SmartPtr. Basically, compilers will > apply EBO once > or twice for a given class, and that's it. The exception seems to > be recent > versions of gcc (I don't remember if its 3.3 or 3.4). > Maybe the implementation can be changed so that compressed_member can inherit from another compressed_member, so as to make this syntax feasible: template <typename T0, typename T1> struct A : private compressed_member<T0> ::and<T1, 1>::type ::and<T2, 2>::type { } or something similar, resolving to a linear hierarchy of compressed_member's. AFAICS, EBO applicability limitations would be solved that way. An sketch of copmresed_member implementation would be as follows: struct dummy{}; template <typename T, size_t Index = 0, typename Base=dummy> struct compresed_member: public Base { // utility metafunc for adding members template <typename Q, size_t AnotherIndex = Index+1> struct and { typedef compressed_member< Q, AnotherIndex, compressed_member<T, Index, Base> > type; }; // rest of the implementation } Get the idea? Joaquín M López Muñoz Telefónica, Investigación y Desarrollo

1 0