Re: [boost] [boost.log] Using stream manipulators to influence record formatting per sink

Subject: Re: [boost] [boost.log] Using stream manipulators to influence record formatting per sink
From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2010-11-09 14:44:50

On 11/09/2010 12:26 AM, Jamie Allsop wrote:
> So I've been using the previously reviewed log library by Andrey
> Semashev with some degree of success. One thing however has caused me a
> great deal of trouble and I'm sure with a couple of changes would become
> a lot easier.
>
> Basically I have an application that outputs tabular data in the logs. I
> have objects that know how to stream themselves either as an ascii table
> or as an html table. I have a custom defined manipulator that can be set
> for a stream that the objects can check the value of, and stream
> themselves either as text or html as appropriate.
>
> In using the log library I face two major challenges. The first is how
> can I set the manipulator to the desired value so that the log record
> will be aware of the manipulator while building its message? It is not
> possible to do this by calling set_formatter() on the sink since the
> this receives *an already composed* record.message() string.

That's right, record message is composed only once, for all sinks. It is 
intended to be used for informational purposes, if you need a per-sink 
formatting logic then you should use sink formatters and attributes. 
This is a fundamental limitation, I don't think I'll be able to change 
it without severely compromising usability.

I'd suggest attaching your objects as attributes to the log record and 
then put them into the stream either way in the sink formatter. I'd even 
decouple the object you have from the formatting logic but that's not 
strictly necessary. You'll have

   text_sink->set_formatter(fmt::stream << as_text("MyAttr"));

and

   xml_sink->set_formatter(fmt::stream << as_xml("MyAttr"));

where as_text and as_xml will both extract your objects from the 
"MyAttr" attribute in the log record and then apply the appropriate 
formatting. No need to imbue the stream with anything.

> So starting at the beginning I have objects that implement streaming
> something like this:

[snip]

> Given that it is
> generally desirable to treat different sinks differently (and often the
> reason they exist) it would be much nice if there was the concept of
> sink-level attributes. These could be added to the record's attribute
> set along with the source and global attributes.
>
> Currently there is no way (it seems) to associate any state with a sink
> or make any decision during logging that relates to a sink. We can set a
> custom formatter that applies to the log message created but that has no
> effect on the record message itself. (Decorators only seem to be for
> post-processing of the record message). We can also set a custom filter
> to determine whether a log record gets logged by a given sink. It seems
> odd then that we have no way to to be able to say (for example) that a
> given sink will have an attribute that indicates that all messages
> through that sink should be in a certain format (for those objects that
> support the formatting). From looking at the code this does not appear
> to be a major issue so hopefully it could be supported.

I think you misunderstood the relations between sinks, filters and 
attributes. There can not be a sink-related attribute because it is 
attributes (or, more precisely, their values attached to the given 
record) decide which sinks will get the record. This decision is made by 
filters. Therefore each sink gets the same set of attribute values and 
at that point the set is fixed. It is the sink's nature, formatter 
included, what makes the same log record look different in files from 
different sinks. So you could maintain some kind of attributes in sinks 
but they wouldn't be able to participate in filtering and be injected 
into the record somehow, so they would be functionally equivalent to 
different sink formatters.

> It would also be highly desirable to be able to specify manipulators
> directly that could then be applied to the record message stream itself.
> Supporting this at the global and sink level should be possible. If we
> had sink-level attributes the technique I described could be used to
> simulate this, even though it is somewhat kludgey.

One of the key ideas behind the library was to isolate the level of 
emitting log records and the library configuration level. Any given log 
record may potentially be processed by any set of sinks and the code 
that emits the record has no idea about which sinks and how will process 
it. So any processing nuances should be described in the library 
configuration code (in your case, I believe, sink formatters is the 
right place). I'd like not to expose any sink-related interaction to the 
user that wants to simply write something into log.