Re: [boost] Re: Re: Profiling Library suggestion

From: christopher diggins (cdiggins_at_[hidden])
Date: 2005-02-11 20:14:25

Christopher Diggins
Object Oriented Template Library (OOTL)
http://www.ootl.org
----- Original Message ----- 
From: "Preston A. Elder" <prez_at_[hidden]>
To: <boost_at_[hidden]>
Sent: Friday, February 11, 2005 4:34 PM
Subject: [boost] Re: Re: Profiling Library suggestion

> Hum
>
> I don't know why, but my last e-mail on this subject went AWOL, so I'm
> going to re-type it (which is a shame, as it was quite long).

:(

> I think the profiler is making great progress, however I still have two
> major issues with it.

Thanks.

> First, that the policy stuff is all static.  This makes it impossible for
> me to have two or more individual profilers going, using the same policy,
> but, say, a different output file, without making the policy extremely
> complex and doing all kinds of string matching to sort out whats what.

How would you propose doing it differently?

In the latest version You can now use meta-data to associate extra tags with 
data to help for generating different reports. Consider the code at 
http://www.cdiggins.com/profiler/v2/custom_profile_manager.hpp , one can 
introduce a new field into meta_profile_data, which would hopefully be 
satisfactory for grouping profiles?

> I want the 'huge', 'big' and 'small' profile instances to use different
> collectors that all use the same policy (accumulate and count, plus print
> to file every, say, 30s), but, say, output to different files.  With the
> static nature of everything you have in there regarding this, I would need
> to either:
> a) create a separate policy just to change the output file.

Would this be so bad?

> or b) in the policy, some how determine which named entries go to which
> accumulator, and then record as such.

And now c) add extra metadata and switch on it.

> Having them separate would allow me to disable them separately, too.

One solution for disabling groups of profiles is to define multiple profile 
managers for the different profile groups:

struct huge_profile_manager : default_profile_manager { }
struct big_profile_manager : default_profile_manager { }
struct small_profile_manager : default_profile_manager { }

profiler<huge_profile_manager> p1;
profiler<big_profile_manager> p2;
profiler<small_profile_manager> p3;

// turn off only big_profiles
template<> struct profiler<big_profile_manager> : empty_profiler { };

(note: empty_profiler doesn't exist yet, but I will add it).

> For
> example, I could use the 'huge' to get a general idea of what section of
> code on average takes the longest.  Then I could drill down slightly
> further to find out which portion of that code is taking the longest, and
> then even further to find out which particular options are taking longest,
> and I could do it all individually.
>
> Second is the syntax, which requires me to either declare all profiled
> sections ahead of time, pause them, then when I get to the part I want
> resume and pause them again, or use ugly scoping syntax to encapsulate
> multiple blocks of code.
>
> For an example of the first, assume in the MyFunc I want every opX_*
> call profiled as one.  The goal here is to find out, over calls to MyFunc,
> how much time opX_* is taking, not how many times opX_* is called and the
> average time for that call.  Like so:
>
> void MyFunc()
> {
>    Profile p_op1("op1"), p_op2("op2"), p_op3("op3");
>    p_op1.pause();
>    p_op2.pause();
>    p_op3.pause();
>
>    for (i=0; i<100; ++i)
>    {
>        p_op1.resume();
>        op1_part1();
>        op1_part2();
>        p_op1.pause();
>
>        p_op2.resume();
>        op2_part1();
>        op2_part2();
>        p_op2.pause();
>
>        p_op3.resume();
>        op3_part1();
>        op3_part2();
>        p_op3.pause();
>    }
>
>    do_something();
>
>    for (i=0; i<100; ++i)
>    {
>        p_op1.resume();
>        op1_part1();
>        op1_part2();
>        p_op1.pause();
>
>        p_op2.resume();
>        op2_part1();
>        op2_part2();
>        p_op2.pause();
>
>        p_op3.resume();
>        op3_part1();
>        op3_part2();
>        p_op3.pause();
>    }
> }
>
> This is the only way to get all opX_* functions accumulated together, so
> that the count goes up by 1 for each call to MyFunc, and the time for all
> instances of opX_* is added, as opposed to getting a 200 count opX_*
> function calls, and then the averaging being the time per call (which is
> not what I want in this case).

Why is this code so bad and what is the alternative?

> For an example of the second, try this:
>
> void MyFunc()
> {
>    {
>        Profile op1("op1");
>        op1_part1();
>        int rv1 = op1_part2();
>    }
>
>    {
>        Profile op2("op2");
>        op2_part1();
>        op2_part2();
>    }
>
>    if (rv > 30)

rv is not accessible here.

>        Profile::generate_report();
>
>    {
>        Profile op1("op1");
>        op1_part1();
>        op1_part2();
>    }
> }
>
> Obviously, in the example, I want the profiling data done before the
> report generate in the report generated.  And the stuff afterwards not.
> So I can't, as in the previous example, declare it at the beginning.
> Either way, the scoping operators used here are kind of ugly.

I don't follow your intent here sorry.

Thanks for your feedback,
CD