Re: [boost] Re: io operations for stl containers?

From: Terje Slettebø (tslettebo_at_[hidden])
Date: 2003-02-24 22:56:41

>From: "Jason House" <jhouse_at_[hidden]>

> Terje Slettebø wrote:
> >
> > >From: "Vladimir Prus" <ghost_at_[hidden]>
> >
> > Sorry for having taken so long to respond to these messages. I felt a
need
> > for a break, to consider how it might be done.
>
> I was wondering about this line of discussion earlier today... wondering
> if it died on the vine or not.  I'm glad to see that it hasn't

Right. After I had sent it, I found that "break" wasn't quite right. I meant
that I had been considering it, including what had been said in the thread.

Specifically, like I said in an earlier posting, I was wondering if there
was enough commonality to warrant a library implementation. Then I happened
to look more into Boost.Tuple, and realised that even a simple solution
could be useful, as you and others have pointed out, as well.

> > In a way, something good came from it, as well: I've recently looked at
> > Boost.Tuple, and I see that they have I/O operators defined (in
> > tuple_io.hpp).
>
> Well, it definitely seems like the tuples were thinking along similar
> lines with a start, middle, and stop delimiters.
> The documentation brings up a point about parseability of data streams.
> It doesn't quite make sense to me that there should be restriction to a
> single character in order to to make things uniquely parseable.  If it's
> a fixed sequence of characters, I don't see how that makes it any
> significantly less parseable...  Maybe I'm missing something?

The docs says:

"Note that extracting tuples with std::string or C-style string elements
does not generally work, since the streamed tuple representation may not be
unambiguously parseable."

It's not about the delimiters. which it seems you mean, but about the tuple
elements. Consider:

tuple<std::string,int> test;

stream >> test; // Stream contains "a string, with comma, 123"

Here, it can't know that the first comma is part of the string, and not
meant to separate elements. Not to mention that it would stop after "a", due
to the space character.

I agree that it should be possible to have multi-character delimiters,
without creating parsing problems. It could be an idea to keep this, as it
may make for more flexible output. Even single-character delimiters is quite
flexible, as you can even get each element on its own line, by using '\n' as
the element separator. To look again at one example of how the tuple way
might work:

std::vector<std::pair<char,int> > test;

std::cout << test: // Using defaults

(('A',1),('B',2),('C',3))

In this case, it's not possible to set the format for each type separately.
Maybe it could be good to keep that feature of the current composite
operators, as well, something like:

typedef std::pair<char,int> map;
typedef std::vector<Map> map_list;

map_list test;

std::cout << io::set_delimiter<map>("\n") << test;

Output:

(('A',1)
('B',2)
(C,3))

Or, generating program-code like listing:

std::cout << io::set_format<map_list>("{\n","}\n",",\n")
          << io::set_format<map>("{","}",", "}
          << test;

{
{'A', 1},
{'B', 2},
{'C', 3}
}

Maybe also:

std::cout << io::set_format<map_list>("(\n",")\n","\n") << io::set_indent(2)
<< test;

(
  ('A',1)
  ('B',2)
  ('C',3)
)

You might also have a non-template overload of the manipulators, which sets
the delimiters for all types, as done in tuples.

This may also have a positive effect on a serialisation library: The
standard types will have default stream operators.

> It does enable defaults and allows a way to customize each spacer
> individually, which is a good addition.  I think the ability to set all
> 3 also is a must-have :)

Yeah, I think that's useful, too. :)

> The tuple functions, as provided should be extremely easy to call from a
> debugger since there is no templating going on.

Actually, there is. They are defined in tuple_io.hpp as:

template<class CharType, class CharTrait, class T1, class T2>
inline std::basic_ostream<CharType, CharTrait>&
operator<<(std::basic_ostream<CharType, CharTrait>& o,
           const cons<T1, T2>& t)

As you say, this may be a problem to call from a debugger, unless it
supports calling function templates. If one need to call it from a debugger,
one could always wrap the code in a function.

> I think that if when
> composite_format matures, there should be a way to add non-templated
> calls to change defaults.
> maybe:
> namespace composite_format{
>   namespace tuple{
>     ... set_open(char x){ return composite_format_open<tuple>(x); }
>   }
> }

The manipulators might also be handled the same way as above, wrapping them
in the function.

> Actually that example doesn't quite work because <tuple> is not well
> defined.  The previous discussion of composite_fromat (or at least the
> code presented) did not allow for a generic class of types, and only
> provided functionality for a very specific type.

Right. There was a suggestion for allowing generic formats, though, using
the same format for all instantiations of a template. The question is how to
do it. The current version stores the format for each specific type, as you
say. Volodya suggested a couple of ways it could be done, partial
specialisation, or storing the template name in a map. However, it could be
hard to use the latter, as it could be hard to find the right type to look
up, when outputting, given that you have specific types, not templates.

Overloading of class templates might have helped, here. E.g.:

std::cout << io::set_format<std::vector<std::pair<char,int> >(...); // Set
specific format (io::format is a class template)
std::cout << io::set_format<std::vector>(...); // Set generic format for
std::vector (io::format overloaded with a version taking template template
parameter, and specialised for std::vector)

If partial specialisation was used instead, you'd still need to specify the
full type (even if only the generic format is set), and then, how to
differentiate between specific and generic format? For example:

std::cout << io::set_format<std::vector<std::pair<char, int> >(...); // Uses
partial specialisation of class template set_format for std::vector, setting
generic format

> > Maybe these routines could be generalised, and used for any
> > composite/compound type, including tuples?
>
> generic type = map, list, tuple
> specific type = map<int,int>, map<int,float>, list<int>, etc...
>
> I should try to think about this more than I have :(
> >From what I have thought about it, allowing a generic type creates room
> for unexpected behavior in output when there are composite types
> containing composite types, and somewhere along the lines a *generic
> type* default is overridden.  There might be a specific reason for a
> bunch of lists inside of a composite type to have a specific set of
> delimiters...  but it probably isn't desired for the lists inside those
> lists to be forced into using the same delimiters.

Right. Well, as mentioned, the current version uses format for specific
types, so in that case, you could format each part of the nested container
as you wanted, as shown with the 2D-array, which of course is an array of
arrays.

You could have any specific format override any generic format, as you say.

> There is also room for trouble when the delimiters for both a generic
> type and a specific type have been set...  I'd think that in general,
> the generic type would be lower precedence.  But what if the generic
> type was set more recently?  Should there be two forms for setting a
> generic type's delimiters?

Hm. I guess we could use some experimentation, here. :)

Regards,

Terje