Re: [boost] Re: formatting manipulator for UTF-8

From: Karl Nelson (kenelson_at_[hidden])
Date: 2000-08-19 18:57:58

[...]
> >
> > (Note, this is quite different from the streamprintf code someone
> > posted.  There they were trying to impose some meaning on the indicators.)
> 
> I guess it's also arguable whether or not it is appropriate to absorb
> nonsense silently, without throwing an exception (e.g. %x for a string). On
> the other hand, your system has the advantage of giving us a non-sticky way
> to specify hex formatting, which Rudiger's did not achieve. It might be nice
> to add this to the boost design.

It isn't really absorbing the non sense in that to the stream it looks
like  "<< hex << string(s) << {restore_state}"  You will notice
in the code that many of the C specifiers fold in because of that.
What is the really difference between a "%s" and a "%c" if you just
care if you want string formatting.  

While this seems odd to accept, it is much easier to implement
this then try to enforce other things like the streamprintf.  After
all we don't care to provide new mechanisms every time a new class
is added or we risk running out of letters.  :-)

  cout << format("%W %Q %d %r") << a << b << c << d;

> > > Okay, so 2^9 is "only" 512 signatures... ;)
> >
> > Actualy 2&9+2^8+2^7+2^6 ... because we will want all the signature
> > combinations below.
> 
> You're right 2^10 -1 is "only" 1024 signatures.

Don't laugh I coded something which did this before.  Sure I needed
m4 to build all of them and it took 10 minutes before gcc decided
it wouldn't overload something that many ways....

> > > Yes, but I don't think that really applies in this case. There really is
> no
> > > excuse for modifying an argument to output formatting, is there?
> >
> > It does actually, because otherwise you are forcing all types to have
> > a T(const T&).  Many classes this may be expensive or perhaps not
> > the intended meaning.  Thus by forcing copy by value you have no options.
> 
> I've been suggesting that Rudiger's code should be modified so that it
> doesn't require copy construction of its arguments. In fact, his test code
> doesn't compile with SGI STL or STLport precisely because of his reliance on
> by-value argument passing (ofstream cannot neccessarily be copied). But it
> would be easy to change the arguments to the basic_format constructors so
> that they were, e.g., const A& instead of A.

Changing that risks causing the compiler to chose between 
  operator  << (ostream& , A&);
  operator  << (ostream& , const A&);

Almost no uers should make code like this but still.
This may not be of concern but it can be a serious limitation.

> > Another problem is the hidden cost of using templates.  That is every
> > set of arguments will need to implement a new function.  Thus
> >   format(s,int(1),int(2))
> > and
> >   format(s,float(1),int(2))
> > share nothing.  This could get to be a very large number of functions
> > for compilers that don't inline and possibly worse very large code
> > for those that do.
> 
> I'm not sure that this cost is any worse than you will get in your case.
> After all,
>     format(s) << int(1) << int(2)
> generates completely different code than
>     format(s) << float(1) << int(2)

Lets assume I use this code 6 times with different numbers of
ints and floats.  In my case it will implement

  operator << (ofrstream&, int);
  operator << (ofrstream&, float);
  operator << (ostream&, format);
  format::format(char*)

That is all no matter how many ints or floats are used.

In the other case, you will get
  format<intt>::format(char*, int);
  format<int,float>::format(char*, int, float);
  format<int,int>::format(char*, int, int);
  format<float,float>::format(char*, float, float);
  format<int,float,float>::format(char*, int, float, float);
  format<float,int,float>::format(char*, float, int, float);

In other words every set of arguments to a format will make
another big while and switch statement which even if marked
reusable (not inline) would never be able to do so. 

Templates which deal with large numbers of types mean making
loads of instantiations as they are used, which for any reasonably
sized program can quickly spiral out of hand.  Again you will
need to judge for yourself is this a problem.

> 
> Moreover, even if you write
>    format(s) << float(1) << int(2)
> many times in the same program, the instances share no code.

Did the argument above address this?

 
> BTW, another (weak, but not totally insignificant) argument for the usage we
> have proposed is that using << to separate arguments typically causes a lot
> more typing than using ,.

typing two char instead of one raises very few problems with me, especially
since it makes it clear what we are doing.  << means shove into a stream
and since I am just pretending that after the format statement we still
have a stream,  it seems like a benifit to show that to the user.  

> 
> > <dreamland>
> > If C++ wanted to deal with this problem they would allow specifications
> > in the template type.  Thus rather then having to define
> >
> >   template <class T1, class T2, class T3>
> >     void foo(T1& t1,T2& t2,T3& t2) {bar(t1,t2,t3);}
> <snip>
> 
> I guess I still don't see how this applies to our scenario.

Addressed in separate reply.

> >
> > Well, it will be a challenge for me because the current compilers
> > I target (gcc, egcs) don't have charT streams yet.   :-)
> 
> I'm sure you know, but you can get a (much more) conforming library...
> www.stlport.org

No I was unaware but then as my users would have to install it as 
well I don't consider it a plus to increase the dependency base. 
If I can possibly use the crappy STL which comes with people compiler
I do.  

> > Just saying that rahter then going for the full charT specification,
> > writting my class for ostream and wostream would likely cover 95%
> > of all users.  Assuming STL gets implemented better in GNU world,
> > a charT version could be made.
> 
> Ah. Yes, that would probably cover 99.9% of all users.

Okay then we agree at least that full charT may not be necessary.  (It
is great if it does.  Which is why boost should use autoconf to 
figure out what their STL has and select the most appropraite code.(

> > Well I was targetting to make as close to a manipulator
> > as I could.  Since "hex" affects arguments after it, I expect
> > format to do the same.  Although this isn't nearly as easy to acomplish
> > as one might hope.  Mine should do an okay job of hiding that it isn't.
> 
> I don't know if it's a good target to pick. I think maybe it makes your
> design more complicated and reduces the clarity of code which uses it.

I have a feeling that my code actually came out simpler for it.  
I tried at first to make format a proper manipulator and add the
extra traits to the stream and all.  BUt my STL was far to brain dead
and just selecting the proxy works fine.

It does rule out this code which I have very little simpathy for.

  (cout << format("foo") ).precison(5); 

(this is an error because the () clause is an ofr_stream not a real
one.)

> 
> > Really though I think that the template one is a show stopper and
> > thus manipulators are much more likely to work.  After all streams
> > take one item at a time.
> 
> So far, I don't see any show stoppers here. I guess I'm confused.

I am making too big of deal about the selection of const here.  
I forgot that most of the time you won't care to pass the types.
Though some part of the argument remains.

> > cout << format("%f %.3f %f") << precision(5) << a << b << c;
> >
> > would place b with precision 3, a with precision 5, and c with
> > whatever the stream default is.
> 
> I'm beginning to be convinced that there's some value in your approach to
> this part. I'm not sure whether that's just because I don't have the special
> non-sticky manipulators I described earlier. Possibly not; your syntax is
> much less verbose. On the other hand, it's harder to remember the meanings
> of things.

True, but like all things in life this is just one of those things that
one has to memorize.  Something which is hard to remember but really 
commonly used (printf specs) means you can go almost anywhere to look
them up.  Having to refer them back to your docs to find your particular
variation means more doc and more learning on the users part. 
Further, mine cuts out the need to label the order of the items unless
reordering is required.  It is a minor savings but still worth it.  

Another nice advantage is that the translator doesn't need to know this
is C++.  They could be given a mixed set of strings from a C program and
a C++ one and he could use the same format statements for both.  So
from the i18n were the strings and the code are separate matters this is
really a win to just use the standard.  (even if the standard is a pain.)

--Karl