From: Scott Woods (scottw_at_[hidden])
Date: 2007-04-10 19:12:36
Hi Gareth and others,
----- Original Message -----
From: "Gareth Buxton" <gareth.buxton_at_[hidden]>
Sent: Monday, April 09, 2007 9:13 PM
Subject: Re: [boost] A meta-proposal for logging
> Hi Andrey,
>> You see, in your example the filter would get "maskable& line", not
>> "line_number& line" as the logging library doesn't know about
>> "line_number" class. This means that you will have to make some type
>> dispatch to get the real attribute type, and I'm afraid this will be
>> a performance killer.
What constitutes death? i.e. how do you know what number of additional
CPU cycles would change the mind of a potential adopter.
There is no real boundary. It is way too dependent on the circumstance.
> It could be but I don't think it necessarily has to. My solution would
> be to tightly couple the filter with the attributes that are to be
> filtered on the basis that if the user is adding the attributes then the
> use knows how best to filter based on them.
> I tend to think that some amount of coupling is inevitable, for the most
> part I would imagine attributes would be well known, frequently used
> types like time (time_t), line_number (int) etc. and would be provided
> as 'standard' in the library.
I would have liked to respond to the individual points raised here but there
are a lot of them.
If there is something useful I can add it is most likely a brief summary of
a logging system recently completed. I balanced many of the issues
* Logging is a stream of mixed-type records or discriminated variants.
* In application code, logging looks like;
log_channel << "Reached here";
log_channel << file_IO_problem;
log_channel << server_snapshot();
log_channel << stack_trace();
* The stream of objects is asynchronously queued, sent over a
network connection, collated, time stamped and stored using a
proprietary format and file management scheme.
* A dedicated viewer allows entry of arbitrary filtering expressions
that typically results in a window-per record type, i.e. a fixed set
* Users can navigate inside the filtered view by entering further
search criteria or by clicking on an auto-generated time breakdown
of the view.
* The record types are "user defined". Registration of a type includes
enough information for the viewer to display full attribute information.
* The logging applications (i.e. the source of log records) communicate
with the server using a formalized protocol. An obvious message (or
signal) within this protocol is LOG (i.e. the connecting client is
a log record, an instance of a discriminated variant).
* Other less obvious protocol messages involve the distribution of the
registered types, i.e. the logging application publishes its type
to the logging server.
* The logging server accumulates and merges all the type information from
all the different sources.
* The logging server persists its own "super-symbol-table" to reduce
traffic and also to enable the display of type information when no
logging applications are connected.
This might be termed a "full implementation" of logging. Some high-end
have certainly dragged it a long distance from where it started. Some
useful lessons picked up along the way;
* The software machinery that provides for creation, transport, queueing,
retrieval and viewing of logs is far more important than a particular
set of attributes.
* There is no ultimate set of attributes. The set of attributes on this
after almost every discussion and remains in the state "to be
* Many of the initial requirements were eventually shown to be silly. Even
now it remains
very difficult to convince interested parties of this, e.g. a live
received log records is not useful.
* There are huge benefits in creating a logging system that transports and
blobs (mixed-type log records). All the significant related software is
independent and inherently "forward compatible".
* The cost in software complexity and dependence on formalized communcations
mechanisms is non-trivial.
* Given another opportunity and certainly in the case of a Boost.Log
would probably swing away from the "published type info" approach and go
text. This is mainly due to the amenable nature of text (much easier to
get a google search
implemented) and general software simplicity. The costs include space
resources (RAM consumed
in log queues and disk space) and some loss of viewing+filtering
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk