[boost] Re: logging lib, v1.3

From: Darryl Green (darryl.green_at_[hidden])
Date: 2004-11-03 04:08:58

Johan Nilsson <johan.nilsson <at> esrange.ssc.se> writes:
> Could each log provide its own log item definition? The idea just popped up,
> so I don't know if it's good or bad.

Well, potentially I guess the "stream" type could be per log, but then I think
there would still be a need to convert any/all of them into a single "string"
type. To save the ievitable confusion from using string and stream to mean
something completely different, perhaps relabel as follows:

_stream_ becomes _message_builder_. That is, it is the object that allows data
to be logged as a single message to be assembled into the message. It could be
an ostringstream, data added using operator <<, the message being the underlying
string. It could be an ACE_LogMsg or whatever the struct is.

_string_ becomes _message_. A message is the thing you deliver to an appender
(and possibly modify along the way - though I wonder if perhaps the modifier
would actually be better off operating on the message_builder BEFORE it is
turned into an (immutable) message.

We could conceivably (but compile time expensively?) allow for a variety of
message_builders. It would have to be possible to generate a single message type
from all of them though. Then the appenders would need to be able to actually
generate something useful from it. Stuffing a whole heap of different sorts of
"messages" into a message type of boost::any isn't going to make output easy :-)

> > If that isn't enough, as I've said before, it is possible
> > to have some completely custom "string" and "stream"
> 
> That's good, but it doesn't put off the formatting (from whatever to text)
> so that it can be performed in the background.

Its a question of definition of "formatting". With the non-custom version the
"whatever" must be converted to a sequence of chars that some formatter can
further process. With some other message type (not a string) it is clearly
possible to delay (possibly to some completely offline processing if you want a
binary log file) the formatting.

> > I don't think actual
> > implementations of a wide variety of these options need be included in the
> > library, though over time it could be extended with additional submissions.
> 
> No, but I strongly recommend that enough thought go into this process, to
> avoid the initial implementation preventing this without major rewrites.

Agreed. Thanks for the help in doing that ;-)

> > Appenders really only need to receive a character iterator range.
>
> You are assuming appenders only need to work with pre-formatted text.

Well - some form of sequence of "chars", but I take your point. I really meant
to suggest that for the case where the message is in this form, perhaps a type
other than string could be used. I agree with you that some completely different
message type should be allowed for. Everything except for the low-level
mechanics of an appender (writing to a file or whatever) is coupled to the
message type, and very little else in the library. So I think it is reasonable
to provide/allow completely separate sets of appenders for particular message
types. It might be possible to provide some simple appenders such as writing
raw messages to a file or socket as simple templates. Others - like a
windows event log appender - appear to be far less generic, or at least
require some form of formatting/converting policy peculiar to the message type,
to be provided.

> 
> > However, more wildly different types should be possible and at least an
> > example provided (eg. some form of sequence of attribute/value pairs -
> maybe a
> > map<std::string,boost::any> would be a good example?).
> 
> That sounds interesting - even though I'd like to stay out of boost::any /
> boost::variant dependencies also. NIH syndrome or minimizing dependencies?

My suggestion of its use in an example is a combination of laziness (in some
ways) and a clear demonstration of flexibility. If it actually performs well
enough to be useful to people in its own right it qualifies as an experiment
that paid off, otherwise it is a prototype for something perhaps less flexible,
but more efficient? I'm assuming that an appender for this message type would
have to be quite heavily templated, and pull out the fields it was interested
in into their original types, by name, then format them. Whether this has any
utility remains to be seen...

> Also, I've got a feeling that at least boost::variant is pretty heavy on the
> compile times ... am I right?

I'm not an expert on those libraries. I haven't even played with variant yet.
Any shouldn't be too bad at compile time - runtime performance might be an
issue though!

> >
> > I actually think the current proposal goes a long way towards meeting all
> of
> > these (there are some areas that need "fixing" but I think the basic
> design is
> > there). Are there any other design issues that need to be resolved or is
> it only
> > a case of the implementation having a few gaps/bugs?
> 
> - The actual logging calls themselves also should be made as efficient as
> possible, not only the enabled/disabled checking.

Yes. I meant the requirement re scaleability to cover (some aspects) of that.
The trouble with "efficiency" is that means different things to different
people, especially when you start trading it off against other features. Are
there any specific aspects of performance/efficiency you are concerned about?

> - Localization support (perhaps not so basic).

Good point. As I thnk you said in another post, that seems to be one feature
that would be directly facilitated by some form of non-string message
representation. However, I can also imagine a system where the localization
formatting was done by the message_builder as the log message was generated,
not on output. Do you have cases where different appenders should apply
different localization? If localization is too expensive to do "up front" are
you sure it isn't practical to do some cheaper form of "serialisation" up front
and do the localization offline?

Regards
Darryl.