Boost logo

Boost :

From: Darryl Green (darryl.green_at_[hidden])
Date: 2004-07-01 02:04:25


John Torjo <john.lists <at> torjo.com> writes:
>
> Dear boosters,
>
> Worked some more on the logging library

Hi John,
I really like your idea of simplifying the actual logging by not directly
modeling the hierarchy. The if/else macro to only conditionally evaluate the
arguments is also an essential feature that your design includes. However, I
think one significant feature is missing, and some aspects of the design and
implementation don't seem to suit at least one use (mine of course :-).
However, logging seems to be one of those things that everyone at least thinks
they have special requirements for, and maybe I'm missing how to apply the
library effectively. I think a *small* library is desirable, and any usage
dependant issues should be dealt with by making the library
extensible/configurable, not larger.

The specific features I'm concerned about...

Support for dynamic log creation:

Some form of dynamic log creation would be very useful. Consider a networked
system of some kind in which the application instantiates an object with a
runtime determined name/identifier (such as the address/name of the
peer/client/server/node/service/entity). So to turn on detailed logging of
information from just one such entity, named "bob" one would (via an
appropriate runtime UI) enable_logs("app.trace.client.bob"). The object that
represents a connection with bob needs to hold some scoped_log object. The
actual writing to the log would need to be different for a dynamic vs static
log - a macro taking the scoped_log rather than the log_name.

Possible bug in handling of early logging:

Isn't the effectiveness of the facility to buffer data written prior to
defining the log dependent on the order of instantiation of the static objects
that log vs the instantiation of the logid's themselves? If any logging is
attempted after the logid name is registered but before any log_funcs have
been added the messages will be lost. Or am I missing something here?

 
Log enable/disable/add/remove performance/scalability:

The separation of logging and the implied hierarchy has made for a relatively
expensive update process when the logging settings are changed. As you point
out in the docs, this shouldn't be a big issue, as they don't change often.
However, I am somewhat disturbed by what looks like an O(n^2) cost for log
enable/disable. Maybe this is just an implementation detail that can be
optimised relatively easily (couldn't the raw enabled array be implemented as
a map keyed by prefix?). One could also imagine building a more elaborate,
data structure to represent the hierarchy if this level of optimisation was
really needed (I'm not suggesting that you should do this).

Multi-threading performance:

The caching of the loglib_info on a per thread basis seems to have the
potential to produce some large latencies in the operation of a heavily-multi-
threaded application whenever logging settings are changed. The loglib_info is
certainly not a lightweight object and is copied between threads (while
holding a lock on the master/global instance). In addition to the copying,
just the storage used per thread due to the duplication of data may be
significant for an application with a lot of lightweight threads.
I'm also not convinced that your lock-free counter is strictly portable, but
in practice I don't know of a system it wouldn't work well enough on.
Regardless, I think it is solving the wrong problem (at least for systems with
any form of real-time constraints) in that the whole design seems to offer low
average latency, but with a worst-case latency that depends on and scales
poorly with the size of the system (measured by number of logs and number of
threads). Finer-grained locking would seem preferable.
A version that explicitly forgoes thread safety would also seem useful - and
not just for single threaded apps. It seems like a lot of the implementation
is tied up in the thread support area - it would be nice to separate the
threading issues more cleanly.

Efficient long/complex log entries:

>From the docs:
// to avoid this:
// (which could be very time-consuming - acquiring the log one million times)
for (int idx = 0; idx < 1000000; ++idx)
    BOOST_LOG(app) << "message at idx " << idx << std::endl;

Not only time consuming - potentially, only some lines of output would be
generated (if log settings changed) and other log output could be interleaved
between lines of this message.

// you will want to do this:
// (very efficient)
if (BOOST_IS_LOG_ENABLED(app)) {
    std::ostringstream out;
    for (int idx = 0; idx < 1000000; ++idx)
        out << "message at idx " << idx << std::endl;
    BOOST_LOG(app) << out.str();
}

Much better - but then why can't I do this:

if (MAKE_LOG(log, app)) {
    for (int idx = 0; idx < 1000000; ++idx)
        log << "message at idx " << idx << std::endl;
}

Which should be more efficient again?

Regards
Darryl.


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk