Re: [boost] [log] Release Candidate 4 released

Subject: Re: [boost] [log] Release Candidate 4 released
From: vicente.botet (vicente.botet_at_[hidden])
Date: 2010-01-30 15:54:22

----- Original Message ----- 
From: "Andrey Semashev" <andrey.semashev_at_[hidden]>
To: <boost_at_[hidden]>
Sent: Saturday, January 30, 2010 8:56 PM
Subject: Re: [boost] [log] Release Candidate 4 released

> 
>>> There is a single lock-free queue per sink frontend.
>>
>> Have you considered to have one queue by thread, this could reduce
>> contention as source threads will not share the same queue. In this
>> case the kind of queue you need has only one producer thread and one
>> consumer thread, which has more efficient lock-free implementations
>> (at least this is what I have heard).
> 
> Yes, I have considered this idea. IMHO, it won't give much performance 
> benefit, compared to a single lock-free queue, unless all your 
> application does is logging alone, from dozens of threads. I did not 
> experiment, though, so I don't have numbers behind myself.
> 
> I have plans of redesigning the queue in order to allow to limit its 
> growth, which may be required in heavily loaded applications. 

This is a good idea. In this kind of context it would be worthwhile to have some mechanism to decide which log records should be lost, in case there is not enough place, severity could be an important factor.

> Per-thread 
> queues would complicate this addition.

Why?
 
>>> With this frontend one may order records not only by a counter
>>> (which will eventually roll over and break the ordering)
>>
>> You can define an order that takes care of roll over. It is enough
>> that the distance between two samples don't saturate the counter
>> type.
> 
> Technically, you can't define a reliable distance between counter 
> samples, because it's always possible that a record from a low-priority 
> thread will have the counter value beyond it.

You are right, there is no reliable distance. But in real cases, how many logs can be done before this low-priority thread runs? If you consider this unreliable you can mix time stamps and sequence counters. The distance of two sequence counter having the same time should be quite small.

>>> but by any attribute, time stamps for instance.
>>
>> On some platforms time stamp it is not enough, as several logs can
>> share the same time stamp when the clock is not enough precise. You
>> will need in addition a sequence counter to get a complete order.
> 
> Well, from the observer's point of view, records that have the same time 
> stamp were emitted simultaneously. It's undefined which of them happened 
> first, thus no point to define their order in log. One might want to 
> improve the time stamp precision to get a more crisp picture of 
> inter-thread relations.

This is thru when the logs are done by different threads, but not when they are doen by the sme thread. If the user do

log A
log B

The user expects A is logged before B even if these logs share the same time stamp.
  
> However, you are free to order records as you will. You can define the 
> ordering based both on the time stamp and the record counter.

I agree. I have to say only that I would expect the library provides this by default.
 
>> IMO what the user needs is an complete order related to the time
>> point where the log was sent. I don't see other orders that could
>> interest the user, do you have some examples?
> 
> You can't implement a complete order, just the order within an ordering 
> window. But yes, the main aim of this functionality is to maintain 
> chronology of log records.

So, do you think it is worth opening this order to the user? Does the library provives at least one order that maintain chronology of log records? If yes, this should be the default behavior. If I have undestood the user needs to state explicitly an order, isn't it?

Regards,
Vicente