Re: [boost] [boost.log] Using stream manipulators to influence record formatting per sink

Subject: Re: [boost] [boost.log] Using stream manipulators to influence record formatting per sink
From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2010-11-10 13:04:52

On 11/10/2010 01:39 AM, Jamie Allsop wrote:
> Andrey Semashev wrote:
>> On 11/09/2010 12:26 AM, Jamie Allsop wrote:
>>
>> That's right, record message is composed only once, for all sinks. It
>> is intended to be used for informational purposes, if you need a
>> per-sink formatting logic then you should use sink formatters and
>> attributes. This is a fundamental limitation, I don't think I'll be
>> able to change it without severely compromising usability.
>
> I guess that's my point. There is no ability to format the actual record
> message itself and I'm suggesting there should be. From the current
> design this is made impossible but I'm suggesting that perhaps it can be
> accommodated. Perhaps if I don't overload the meaning of attributes and
> focus only on manipulators I might make myself clearer. What I am
> suggesting is really two things:
>
> 1 - Allow for the possibility of passing manipulators down to the
> temporary stream object that is used to create the record message.
> Perhaps this might mean you need to add some sort of manipulator
> facility much like your attribute facility. This would then allow you to
> write your formatters in such a way so as the record message could be
> composed using a stream with the appropriate manipulators present.
>
> 2 - The second thing I am suggesting is that manipulators also be
> supported per-sink. This seems to be a natural extension of what sinks
> do, they decide if a record should be written or not and what format the
> overall message should be written in. In order to support that you would
> of course have to compose the log record message lazily once for each
> sink with different manipulators, but only once, as we do now, if all
> sinks had either no manipulators or the same manipulators. Not trivial
> but not too complex either.

I'd say that it is #2 that is necessary to achieve your goal, since the 
necessary manipulators or whatever can be applied in the formatter, 
which is sink-specific already.

Regarding lazy formatting, I considered this approach when I started the 
library. The main drawback of this approach (which led to its rejection) 
is that lazy expressions complicate logging in non-obvious ways. 
Consider this expression, for instance:

   BOOST_LOG(lg) << my_obj;

Normally you would expect that my_obj is inserted into a stream here, 
using a possibly user-defined operator<<. However, would that be a lazy 
expression, my_obj would have been copied here, possibly several times 
(proportionally to the number of streaming operators in the expression). 
There are other issues, such as compilation times and code bloat. I 
understand and share your reasoning, but I believe that for the most 
cases the current solution is optimal. The need to customize the record 
message is quite rare.

However, I think the solution is possible, either by providing an 
alternative logging facilities (macros, loggers, stream placeholder, 
etc.) to implement lazy formatting, or by using some hybrid solution 
(e.g. by inserting some placeholder into the stream and later replacing 
it with the post-formatted objects). Either way, this would require 
modifying the library (mostly the log record class) and it's not 
completely clear to me what it would look like afterward. Meanwhile you 
can use attributes to achieve your goal.

>> I'd suggest attaching your objects as attributes to the log record and
>> then put them into the stream either way in the sink formatter. I'd
>> even decouple the object you have from the formatting logic but that's
>> not strictly necessary. You'll have
>>
>> text_sink->set_formatter(fmt::stream << as_text("MyAttr"));
>>
>> and
>>
>> xml_sink->set_formatter(fmt::stream << as_xml("MyAttr"));
>>
>> where as_text and as_xml will both extract your objects from the
>> "MyAttr" attribute in the log record and then apply the appropriate
>> formatting. No need to imbue the stream with anything.
>
> Hmm.. this is an interesting idea. Can you give me an example of what an
> actual log statement might look? I'm not visualising this.
>

The attribute can be set in a scope guard manner like this:

   BOOST_LOG_SCOPED_THREAD_TAG(
     "MyAttr", typeof(table_object), table_object);

That way you'll put your table_object as a thread-specific constant 
attribute. There is a similar macro for the logger-specific attribute.

Now the log statement is not crucial and can be as simple as that:

   BOOST_LOG(lg);

Of course, if you want a message accompanying the record, you can add one.

These two macros can be wrapped in a more concise macro like that

   #define MY_LOG(lg, table_object)\
     {\
       BOOST_LOG_SCOPED_THREAD_TAG(\
         "MyAttr", typeof(table_object), table_object);\
       BOOST_LOG(lg);\
     }