Re: [boost] logging

Subject: Re: [boost] logging
From: Espen Harlinn (espen_at_[hidden])
Date: 2011-07-16 14:11:53

I'm having a few thoughts on a "simple" logging library - so for what it's
worth:

I think that it would be useful if a logging library relied on a minimum of
c++ features that is not likely to "break" between platforms and compiler
implementations, as one of the more valuable "features" would be the ability
to log when things doesn't work as expected. 

While I really like libraries like log4j/Log4Net - I've also used ACE
extensively, and I've been quite satisfied with the logging facilities
provided by ACE.

One way of using ACE_Log_Message enables me to log directly to a local
logging service, which forwards log records to another service running on a
central host for further processing - this ability is fairly high on my
"requirement" list. 

Tracking problems with a distributed solution, is simplified when logging is
dealt with centrally, and buffered locally.

For a single process, logging is configured on a global or per thread basis,
something that makes more sense in many situations then the approach
followed by log4j and sibling implementations/work-a-likes.

AFAIK ACE does not implement the following "feature" ( but it has a
buffering mechanism, and extensibility hooks, that makes it relatively
straight forward to simulate):
Sometimes it's also useful if the logger follows the processing chain - like
in a multithreaded application where a task may be handed from thread to
thread in a processing pipeline. The log4j model does not lend itself to
this kind of logging - the purpose would be to group output in the log,
instead of interleaving output from parallel tasks/threads. This tends to
make the log more readable.

I'd also like to implement the "log record" as a continuous block of memory
as in <fixed size header><variable length part> where the layout is binary
compatible between platforms and compilers.

Something along the lines of:
enum message_data_type_t
{
  message_data_type_unknown,
  message_data_type_message,
  message_data_type_host_name,
  message_data_type_process_name,
  message_data_type_user_name
};

struct message_data_t
{
 uint16_t size;
 uint16_t type; // a base set given by message_data_type_t, other values are
user defined 
 // Not necessarily byte oriented, just to indicate that data
 // for the message starts at this point. Encoding indicated by
 // log_record_t.encoding
 uint8_t data[0]; 
};

struct log_record_t
{
 uint8_t byte_order; // If set - big-endian, else little-endian
 /* maybe uint8_t reserved, or something, to allow for word alignment? */
 uint64_t time_stamp; // in number of milliseconds since 1/1/1970 ?
 uint32_t log_record_type; // probably something similar to ACE_Log_Priority
 uint32_t message_area_size; // Total size of the variable message area in
bytes
 uint16_t encoding; // codepage for encoding/decoding messages
 uint16_t message_count; // Number of messages
 // the variable area starts here
 message_data_t message_data;
};

This would at least allow for a set of reusable services to be implemented,
facilitating routing of log records.

Each thread will have a logger
A thread may share it's logger with other threads
The main thread creates the "root" logger. 
A spawned thread may be assigned an existing logger, or it creates its own
logger, as a child of the logger for the spawning thread - ideally this
would require a small change to the "thread" library, to ensure that the
logger was the last object cleaned up on thread exit, and a way to pass
information about the logger for the spawning thread. 

The logger for a thread is accessible through a static method, backed by
thread local storage, like: 

  boost::logging::logger::instance()->log(uint32_t log_record_type, char*
fmt,...);
  boost::logging::logger::instance()->log(char* fmt, uint32_t
log_record_type, va_list args);
and
  boost::logging::logger::instance()->log(uint32_t log_record_type, wchar_t*
fmt,...);
  boost::logging::logger::instance()->log(wchar_t * fmt, uint32_t
log_record_type, va_list args);

The above would add a message_data_type_message entry to the variable area -
other entries could be added through an extensible interceptor like chain 

I'm sure I'd be able to implement it in a way that would provide for reuse
by more "advanced" log processors based on features from many of the other
libraries.

Well, it's just a few thoughts based on a solution that I've found quite
useful.

Best regards
Espen Harlinn
Mobile:[phone]
mailto:espen_at_[hidden]