[boost] [logging] Interest check on logging library.

Subject: [boost] [logging] Interest check on logging library.
From: Jason Wagner (jason_at_[hidden])
Date: 2008-12-26 12:48:17

Hello all,

I know there are two proposed logging libraries out there for Boost,  
already, but I had a different itch to scratch on a project.  I think I  
saw that Andrey Semashev was going to submit his far more mature version  
soon, but I thought I'd throw my technique out there for an interest check  
and hopefully commentary.  This is very much a prototype-- there's a lot  
of work yet to be done but the basic outline and functionality is  
implemented.

Getting the code
----------------
http://www.nialscorva.net/boost_logging-0_1.tar.bz2

File is also attached, since it's small.  Just unzip, put on path, and  
include.  It's header only for now.  Tested against 1.36 and 1.37 on gcc  
4+ and msvc 8+.

Elevator Conversation Rundown
-----------------------------
My logging library involves no macros, it relies on using template  
metaprogramming and the compiler optimizer to mostly eliminate logging  
statements that cannot be logged.  It also supports conditional output  
within the context of a single statement, complex filter evaluation and  
chaining, lazy function evaluation, and support for standard ostream  
operators.  All features are easily extensible without macros or hidden  
code.

Requirements
------------
I had several wants/needs that weren't elegantly handled by existing log  
libraries.

- Tagging mentality for enabling logs with in-line conditions.  I'm  
writing a client/server program. I want to be able to write:

   logger << severity::error() << "An error has occurred!"
          << severity::debug() << " packet= " << hex_dump(packet);

When debug is disabled, the packet contents are not logged.   Tags should  
be easy to add and define.  There is common code between the client and  
server and I want slightly different output on each:
   // action, ip, and context are local vars that are defined for  
operator<<(ostream&,T&)
   logger << "Action failed to occur: " << action
          << target::server() << " from ip address " << ip << " with  
context= " << context;

When the code is compiled for the server, the ip and context are logged.   
When compiled on the client, it is removed from completely.  The code is  
compiled every time, so there's no risk of stale code hidden by macros.   
The optimizer removes the strings and dead code later.

- Sink selection based upon filter evaluation.  For example, sending  
severity::warning and above to a console and everything to a file:
   filter_split_action f(severity() >= severity::warning());
   f.on_success(my_console_log);
   f.always(my_log_file);
   logger.set_action(f);

- Chaining filter evaluation.  The client sends certain log events back to  
the server, but also to local sources:
   filter_split_action f(severity() >= severity::warning());
   f.on_success(my_console_log);
   f.always(my_log_file);
   logger.set_action(f);

   // build the server reporter
   filter_split_action to_server(target() == target::client() &&  
send_to_server() == send_to_server::yes());
   to_server.on_success(my_server_sink);

   // chain the server reporter off of the original filter
   f.always(to_server);
Chaining to on_success and on_failure works as well.

- Lazy evaluation of expensive functions.  Only invoke them if they are  
going to be logged.  The library accepts any boost::function that returns  
an ostream compatible type and takes no arguments.  The "lazy()"  
convenience functions create a boost::bind wrapped in a boost::function  
for this purpose.
   int nth_digit_of_pi(int place);
   logger << lazy(nth_digit_of_pi,500000);

- Reconfigurable on the fly (excluding compile-time conditions).  Not yet  
implemented but the groundwork is laid and will be implemented in the core  
via facades that act as handles.

- Hierarchical variable sets.  When looking up a variable for filtering or  
for formatting, climb a chain of parents from  
record->logger->channel->thread->global.

Example
-------
namespace bl=boost::logging;

// defined in logger.hpp
typedef boost::mpl::vector<
     bl::condition::greater_equal<bl::severity::warning>,
     bl::condition::in<bl::build_type::release>
     > optimized_conditions;
typedef boost::mpl::vector<bl::build_type::release,bl::severity::info>  
optimized_tags;
// end defined in logger.hpp

bl::logger<optimized_conditions, optimized_tags> log(sink);
log << "severity::info-- not printed\n"
     << bl::severity::debug() << "debug is not printed\n"
     << bl::severity::warning() << "warning is printed\n"
     << bl::severity::error() << "and so is error\n"
     << bl::build_type::debug() << "but build_type::debug fails the  
condition\n"
     << bl::end_record();

Other samples can be found in lib/logging/examples.

Other
-----
There are still a lot of things in the TODO list.  The best look at usage  
is found in the lib/logging/examples directory.

I've been using speed.cpp to take simple timing information to watch  
performance.  Mostly they are one-off snippets I've been using while  
prototyping this.  There's an excel spreadsheet with numbers in it, but  
generally it takes 3+-0.5 times as long to write to a logger as to cout,  
depending upon sinks, conditions, etc.  A lot of this time is spent in  
variable_set, which is an easy target for optimization.  Whole messages  
rejected by filters take about 1/2 as long as writing to cout, while  
rejection at compile time take 1% to 2% of the time to write to cout.

Currently, there is no multithreading support.  The places that need  
guarding are localized and will be implemented as facades on   
variable_set, filter_action, and sink. Core will be a registry and factory  
for populating variable_set parents and filter_actions on loggers, as well  
as properly putting multithreading and reconfiguration facades on things  
that need them.

I'm not planning on implementing i19n, scoped variables, functor  
variables, or some of the other requirements in the boost logging list  
unless there's interest.

There's more, I'm sure, but I can't write the whole project's  
documentation in an introduction email.

-- 
Jason Wagner
jason_at_[hidden]