boost::format benchmark

From: Samuel Krempp (krempp_at_[hidden])
Date: 2001-07-06 08:00:35

How important are speed concerns, in the design of the format class ?
Of course, the goal of format is type-safe, terse, powerful formatting
accepting user-defined types; and performance is not the main criteria.
People rarely put a print statement in the heart of the most crucial
loop..

But if the format class takes 100 times the time it takes to execute
the equivalent ostream commands, it will be rejected by most of users..

To get an idea of the overhead involved in the current design,
I timed loops of 100,000 formattings  :
format("%p1$#x %2 %3 %1 \n")  % 23 % 45.23 % "Caramba";
with the following methods :
- the format object is constructed within the loop
- the format object is stored and reused at each iteration. ("stored
format"), so as to take aside format-string processing.
- fprintf(nullFile, "%#x %g %s %d \n",23 ,45.23 ,"Caramba", 23);
( the arguments are not positional, so the task is much simpler than
format's )
- nullStream << hex << showbase << showpoint << 23 << dec << noshowbase
<< noshowpoint
          << " " << 45.23 << " " << "Caramba" << " " << 23 << " \n";

[ nullStream (respectively nullFile) is a fstream (resp. FILE*)  directed
to "/dev/null" ]

Results were :

. compiled with optimisation (-O) :
printf time         : 0m 0.732s
ostream time        : 0m 1.659s,  = 2.267 * printf

stored format time  : 0m 3.541s,  = 2.135 * stream
format time         : 0m 6.434s,  = 3.879 * stream
func_format time    : 0m 9.998s,  = 6.028 * stream

. compiled with debug (-g) :
printf time         : 0m 0.742s
ostream time        : 0m 2.399s,  = 3.231 * printf

stored format time  : 0m 9.653s,  = 4.024 * stream
format time         : 0m19.305s,  = 8.049 * stream
func_format time    : 0m26.525s,  = 11.059 * stream

How do those results sound ? good enough, or not quite ?

func_format is rüdiger's previous format class, used via function calls.
It parses the format string once for every argument, so it takes a lot of
time. but that's not inherent to the function approach, which could very
well parse the format-string only once.
BTW, in order to pass 'showbase'/'hex'/.. as arguments, 'const T&' was
not possible..
So here is a kind of case when argument passing through function call is
severely limited.

Leaving parsing time aside (it could surely be optimised, but it is quite
independant from the design of the class, in fact),
I'm conscient that when we dont ask for reordering arguments, nor ' '
printf flag, nor centered alignment, nor ..
we can dump directly to the final output stream (given that it was
actually provided to the format object), and that'd give a "1.1 * stream"
ratio or so. Compared to 2.1 with the current design  (we're not counting
parsing time here).
When counting parsing time, the performance difference can only decrease,
since it's adding up in both versions.
so the ratio between format with direct ouput vs. format storing temporary
strings    is always  <2.

In fact I tested such an implementation, called by
format(cout, "%1 %2 %3") %x %y %z ;
But I consider that ratio 2.1 is good enough for our very general format
object, and dividing time by less than 2 for specific cases is not worth
the extra code and extra syntax.

-- 
Sam