Boost logo

Boost :

Subject: [boost] [boost.log] Using stream manipulators to influence record formatting per sink
From: Jamie Allsop (ja11sop_at_[hidden])
Date: 2010-11-08 16:26:52

So I've been using the previously reviewed log library by Andrey
Semashev with some degree of success. One thing however has caused me a
great deal of trouble and I'm sure with a couple of changes would become
  a lot easier.

Basically I have an application that outputs tabular data in the logs. I
have objects that know how to stream themselves either as an ascii table
or as an html table. I have a custom defined manipulator that can be set
for a stream that the objects can check the value of, and stream
themselves either as text or html as appropriate.

In using the log library I face two major challenges. The first is how
can I set the manipulator to the desired value so that the log record
will be aware of the manipulator while building its message? It is not
possible to do this by calling set_formatter() on the sink since the
this receives *an already composed* record.message() string.

My work-around for this is a lot more complex than I'd like it to be and
I'll detail it here in the hope that it provides some insight to Andrey
so that he might be able to offer a nicer alternative more in-keeping
with his library.

So starting at the beginning I have objects that implement streaming
something like this:

template< class CharT, class Traits>
friend std::basic_ostream< CharT, Traits>&
operator<<( std::basic_ostream<CharT, Traits>& Ostream, const my_class& 
Value )
     if( log::output_format::value_in( Ostream ) ==
log::output_format::text )
         write_text( Ostream, Value );
     else if( log::output_format::value_in( Ostream ) ==
log::output_format::html )
         write_html( Ostream, Value );
     return Ostream;
where 'value_in()' simply retrieves the value from the stream.
Now during logging a temporary stream object is created that must have 
this manipulator applied. What I'd like to do is be able to specify if I 
want this manipulator applied on a per-sink basis. In much the same way 
as I can define a specific formatter for a sink, or a specific filtering 
rule for a sink. Currently there is no way to do that (at least not 
without a *lot* of effort) so I'll address this later. Instead I'll 
consider how we can deal with this be setting this desire globally by 
using a global attribute.
We could write something like this:
log::output_format::type OutputFormat = log::output_format::html;
std::pair< boost::log::core::attribute_set_type::iterator, bool >
         = boost::log::core::get()->add_global_attribute
             (   "OutputFormat",
boost::log::attributes::constant<log::output_format::type> >( 
OutputFormat )   );
Later we can extract this value to decide if we need the manipulator 
applied to the stream that is used to compose the log record message.
We can use a helper object that is aware of all the manipulators that 
are to be applied and have it the log record stream based on attributes 
that represent them. For example, given a class called 'manipulators' we 
can define this insertion operator as:
template< class CharT, class TraitsT >
inline std::basic_ostream<CharT,TraitsT>& operator<<
(   std::basic_ostream<CharT,TraitsT>& Ostream, const manipulators&
Manipulators   )
     Manipulators.apply( Ostream );
     return Ostream;
where apply could be implemented (in our case) as this:
     template< class CharT, class TraitsT >
     void apply( std::basic_ostream<CharT,TraitsT>& Ostream ) const
         boost::log::basic_record<CharT> Record( RecordHandle_ );
         apply_output_format( Ostream, Record );
     template< class CharT, class TraitsT >
     void apply_output_format( std::basic_ostream<CharT,TraitsT>&
Ostream, const boost::log::basic_record<CharT>& Record ) const
         log::output_format::type OutputFormat = log::output_format::text;
             ( "OutputFormat", Record.attribute_values(),
boost::lambda::var( OutputFormat ) = boost::lambda::_1 );
         if( OutputFormat == log::output_format::text )
             Ostream << log::set_output_format(
log::output_format::text );
         else if( OutputFormat == log::output_format::html )
             Ostream << log::set_output_format(
log::output_format::html );
This then requires that the insertion operator is called as part of the 
streaming for any log record. In order to achieve this I need to write 
my own logging macros that do essentially what the severity logging 
macros do already but with the addition of streaming a 'manipulator' 
object at the same time so that the manipulators I want added can be 
applied to the stream that is constructed during that process. Here is 
the key macro that I need to write which gets substituted in a chanin of 
#define MY_LOG_STREAM_WITH_PARAMS_INTERNAL(logger, rec_var, params_seq)\
     for (::boost::log::record_handle rec_var =
!!rec_var; rec_var.reset())\
         ::boost::log::aux::make_pump_stream((logger), rec_var).stream()
<< log::manipulators(rec_var)
The important part is this:
The stream object is created and immediately passed an instance
log::manipulators passing the RecordHandle to the constructor.
Finally then I can have manipulators passed down to the stream that is 
used to create the log record message, and in my case allow the logged 
objects to either stream themselves as text or as html.
A downside of this is that this only works with global attributes. So 
either all log sinks would one format or the other. Given that it is 
generally desirable to treat different sinks differently (and often the 
reason they exist) it would be much nice if there was the concept of 
sink-level attributes. These could be added to the record's attribute 
set along with the source and global attributes.
Currently there is no way (it seems) to associate any state with a sink 
or make any decision during logging that relates to a sink. We can set a 
custom formatter that applies to the log message created but that has no 
effect on the record message itself. (Decorators only seem to be for 
post-processing of the record message). We can also set a custom filter 
to determine whether a log record gets logged by a given sink. It seems 
odd then that we have no way to to be able to say (for example) that a 
given sink will have an attribute that indicates that all messages 
through that sink should be in a certain format (for those objects that 
support the formatting). From looking at the code this does not appear 
to be a major issue so hopefully it could be supported.
It would also be highly desirable to be able to specify manipulators 
directly that could then be applied to the record message stream itself. 
Supporting this at the global and sink level should be possible. If we 
had sink-level attributes the technique I described could be used to 
simulate this, even though it is somewhat kludgey.
Most directed to Andrey, but I'd welcome comments and suggestions from 
others familiar with the log library.

Boost list run by bdawes at, gregod at, cpdaniel at, john at