[boost] Re: logging library v1.1 - with documentation!

From: Darryl Green (darryl.green_at_[hidden])
Date: 2004-07-01 02:04:25

John Torjo <john.lists <at> torjo.com> writes:
> 
> Dear boosters,
> 
> Worked some more on the logging library

Hi John,
I really like your idea of simplifying the actual logging by not directly 
modeling the hierarchy. The if/else macro to only conditionally evaluate the 
arguments is also an essential feature that your design includes. However, I 
think one significant feature is missing, and some aspects of the design and 
implementation don't seem to suit at least one use (mine of course :-). 
However, logging seems to be one of those things that everyone at least thinks 
they have special requirements for, and maybe I'm missing how to apply the 
library effectively. I think a *small* library is desirable, and any usage 
dependant issues should be dealt with by making the library 
extensible/configurable, not larger.

The specific features I'm concerned about...

Support for dynamic log creation:

Some form of dynamic log creation would be very useful. Consider a networked 
system of some kind in which the application instantiates an object with a 
runtime determined name/identifier (such as the address/name of the 
peer/client/server/node/service/entity). So to turn on detailed logging of 
information from just one such entity, named "bob" one would (via an 
appropriate runtime UI) enable_logs("app.trace.client.bob"). The object that 
represents a connection with bob needs to hold some scoped_log object. The 
actual writing to the log would need to be different for a dynamic vs static 
log - a macro taking the scoped_log rather than the log_name.

Possible bug in handling of early logging:

Isn't the effectiveness of the facility to buffer data written prior to 
defining the log dependent on the order of instantiation of the static objects 
that log vs the instantiation of the logid's themselves? If any logging is 
attempted after the logid name is registered but before any log_funcs have 
been added the messages will be lost. Or am I missing something here?

 
Log enable/disable/add/remove performance/scalability:

The separation of logging and the implied hierarchy has made for a relatively 
expensive update process when the logging settings are changed. As you point 
out in the docs, this shouldn't be a big issue, as they don't change often. 
However, I am somewhat disturbed by what looks like an O(n^2) cost for log 
enable/disable. Maybe this is just an implementation detail that can be 
optimised relatively easily (couldn't the raw enabled array be implemented as 
a map keyed by prefix?). One could also imagine building a more elaborate, 
data structure to represent the hierarchy if this level of optimisation was 
really needed (I'm not suggesting that you should do this).

Multi-threading performance:

The caching of the loglib_info on a per thread basis seems to have the 
potential to produce some large latencies in the operation of a heavily-multi-
threaded application whenever logging settings are changed. The loglib_info is 
certainly not a lightweight object and is copied between threads (while 
holding a lock on the master/global instance). In addition to the copying, 
just the storage used per thread due to the duplication of data may be 
significant for an application with a lot of lightweight threads.
I'm also not convinced that your lock-free counter is strictly portable, but 
in practice I don't know of a system it wouldn't work well enough on. 
Regardless, I think it is solving the wrong problem (at least for systems with 
any form of real-time constraints) in that the whole design seems to offer low 
average latency, but with a worst-case latency that depends on and scales 
poorly with the size of the system (measured by number of logs and number of 
threads). Finer-grained locking would seem preferable.
A version that explicitly forgoes thread safety would also seem useful - and 
not just for single threaded apps. It seems like a lot of the implementation 
is tied up in the thread support area - it would be nice to separate the 
threading issues more cleanly.

Efficient long/complex log entries:

>From the docs:
// to avoid this:
// (which could be very time-consuming - acquiring the log one million times)
for (int idx = 0; idx < 1000000; ++idx)
    BOOST_LOG(app) << "message at idx " << idx << std::endl;

Not only time consuming - potentially, only some lines of output would be 
generated (if log settings changed) and other log output could be interleaved 
between lines of this message.

// you will want to do this:
// (very efficient)
if (BOOST_IS_LOG_ENABLED(app)) {
    std::ostringstream out;
    for (int idx = 0; idx < 1000000; ++idx)
        out << "message at idx " << idx << std::endl;
    BOOST_LOG(app) << out.str();
}

Much better - but then why can't I do this:

if (MAKE_LOG(log, app)) {
    for (int idx = 0; idx < 1000000; ++idx)
        log << "message at idx " << idx << std::endl;
}

Which should be more efficient again?

Regards
Darryl.