Re: [Boost-users] yet another logging library

From: Josef Grahn (josef.grahn_at_[hidden])
Date: 2008-06-13 10:37:15

Mojmir Svoboda skrev:
>  - serializes multithreaded code
>    there are sometimes faults that occur only with logging off, because
>    of the serialization by logging library. theser are very nasty :)
>    on the other hand it may be solved by removing all shared rsrcs,
>    perhaps by saving each thread's log to separate sink?

I designed a logging library a while ago that we're using quite
extensively in a performance critical multi-threaded environment, and it
actually does precisely that. Each thread has its own logger which can
be connected to a "sink" specific for that thread.

This approach was chosen after making some observations:

1) Synchronization has a performance penalty.

2) Synchronization might affect the behaviour of the program when
increasing the amount of logged data, potentially leading to heisenbugs.

3) Sometimes you actually never need to merge the streams of events.

4) Since the logged data is used for "external" diagnostics only, the
library should allow the application generating the logs to do as little
as necessary before storing them away. Storing primary measurement data,
in a sense.

The idea was to separate the generation of a log event from the
synchronization, transformation and storage of log events.

In this way, the cost of synchronization of multiple log streams is
moved to a later stage. It can either be done in a designated "merger"
thread inside the application, reading from lock-free queues connected
to the "thread loggers", or each stream can be sent to a different file
or network socket and merged off-line by a diagnostics tool (if needed).

Of course, in the latter case you might get synchronization inside the
standard libraries or the OS instead. On the other hand, this can easily
be amortized, albeit with the risk of loosing unflushed log data in the
event of a segmentation fault.

Best regards,

Josef