[boost] Re: Re: logging lib, v1.3

From: Johan Nilsson (johan.nilsson_at_[hidden])
Date: 2004-10-30 13:20:42

"John Torjo" <john.lists_at_[hidden]> wrote in message news:41828D7B.1080802_at_torjo.com...
>> 
>> On the other side:
>> 
>> - Message formatting always takes place in the caller's context. 
> In ~99% of the time I'd like the messages to be formatted and
> written to their destination in a low-priority (idle) background thread,
> that does not interfere with any other parts of my application.
> This holds true even when I don't use any threads in the application.
> The only time when I would not want do it this way is when the platform 
> does not support threads.
> (I deliberately do not suggest any specific implementation here.)
> 
> (As a side-note, I assume you've seen the ts_appender(), which does the 
> *appending* in a different thread)

I've seen it, yes. 

> 
> That's an interesting idea.
> 
> One problem that I see with the above taking place in another thread is 
> that you don't know when it'll happen. This does not play well with:
> - pre-pending time
> - pre-pending thread ID

I basically was thinking about log "messages" as data structures, consisting of least one message id (possibly consisting of facility, severity and the id itself, similar to COM hresults and VMS status codes) + insertion data. Logging a message would include creating an instance of a log item and storing it in the TSS buffer for later retrieval, formatting and writing by the worker thread. Data such as timestamps, thread id, etc is added at this point and can be selected for inclusion/exclusion in the actual writing on the back-end instead.

Using message id's would also facilitate localization, which would be a good thing for end-user visible logging.

> 
> And probably some other modifiers I'll write or other people will.
> 
> As another side-note, if you really want this, you can write an appender 
> which will modify the original string and then write it to some 
> destination.
> 
>> - I miss priority/severity levels :-(
> 
> I don't like the idea of locking you into some fixed priority/security 
> levels.

Neither do I. I was more thinking in the lines of having the severity level defined as an integer, but additionally provide a set of predefined levels; e.g. DEBUG, TRACE, INFO, WARNING, ERROR, FATAL. Disabling/enabling of logs could be done by setting up either a bit mask or a trigger level.

> You can always create your own "app.err" (error), "app.dbg" (debug), 
> "app.warn" logs, and log directly there (which you can then 
> enable/disable at any time)
> 
>> - I get the feeling that the logging entity always knows where the log messages are destined. 
> I'd like (I think) a single logging "sink" where the messages can be sorted
> at the back-end ("appenders") to provide some looser coupling
> (read the comment at the bottom of this posting though).
> 
> From your point of view, you can still think of it that way.
> 
> The best way to better grasp this IMO is to take a look at the examples.
> I would say that the coupling is as loose as possible ;)

I take your word on that, it's just that I'm trying to grip the concepts.

> 
> Whenever you add an appender/modifier, you don't specify which log it 
> will be added to - you specify a logs_specification, where you can use 
> the * wildcard (same goes for enabling/disabling).

Maybe it's the word "log" that confuses me. For me a log is more the final destination; e.g. the event log, a file, etc ... In this case it is more like a channel for log messages, or?

> 
> For example, to write all errors to cout, you'll say:
> 
> // assuming all error log(s) contain 'err' in their string name
> add_appender("*err*", write_to_cout);
> 
> To turn off debug logging, simply say:
> 
> // assuming your debug log(s) all contain "debug" in their string name
> disable_logs( "*debug*");
> 

The above is exactly one of the things I like about your proposal.

> 
> 
>> - This opinion is probably not shared with others, but I'd prefer the library to be independent 
> from the Boost Thread library. The dependency feels kind of the wrong 
> way around.
> 
> This dependency is only if you use threads yourself.

Naturally, but that alone might be enough to make some people refrain from using it. I'd prefer not having to add the boost thread library to my dependencies simply because I want to log message in the background. Also, there might be special circumstances when you'd like to log messages from system invoked routines, e.g. Win32 completion routines/APCs/console control handlers, Unix signal handlers, VMS ASTs, system thread pool callbacks etc... 

Most of these cases are those where you are normally adviced not to do anything more than absolutely necessary, but occasionally logging from those are not out of the question.

> 
> The library needs to be thread-safe, and it would really see awkward to 
> reinvent the weel. Also, ts_appender() clearly needs it.

Clearly needs Boost.Thread or thread support in general? I've got a gut feeling that this is one of the not-so-rare-cases when duplicating code to lessen dependencies *might* be justified. It would of course be possible to parameterize the threading implementation, but as one of your goals is to minimize compile times if might not be feasible.

Do other libraries use Boost.Thread to implement thread-safety? Should they?

> 
>> - Tests?
> 
> TODO ;)
> 
>> 
>> And a question:
>> 
>> - Did you consider the possibility to integrate with syslog / NT event log?
>> 
> 
> Yes, in the future. Any help is welcome in this area.
> 

I believe that it's not so simple to integrate with the NT event log as the current Boost.log is entirely based on pre-formatted strings. The event log is more based on message id's, insertions strings and message resource dlls, IIRC (long time since I used NT event logging).

// Johan