Re: [boost] [log] Review-ready version in the Vault

Subject: Re: [boost] [log] Review-ready version in the Vault
From: Darryl Green (darryl.green_at_[hidden])
Date: 2009-02-16 07:17:16

On Sun, 2009-02-15 at 16:27 +0300, Andrey Semashev wrote:
> Peter Dimov wrote:
> > Andrey Semashev:
> > 
> >> Yet again, I consider syslog as an API, not just some protocol. This 

     #include <syslog.h>

       void openlog(const char *ident, int option, int facility);
       void syslog(int priority, const char *format, ...);
       void closelog(void);

       #include <stdarg.h>

       void vsyslog(int priority, const char *format, va_list ap);

Now, this API, by itself is problematic. It assumes a syslogd. The whole
idea of a syslogd assumes a security modem where opening low numbered
ports requires root priveledges. The openlog options are largely
platform dependent as well. What it reduces to is a C printf style
interface to generating textual log message bodies, the only work done
behind the scenes that is in any way portable is to prepend to the
resulting string the facility and priority information and to send this
to the mysterions syslogd by a mechanism not defined here. In practice,
the way this is done is by taking the resulting message and sending it
over a unix domain socket to /dev/log.

> >> API is also standardized and can be used outside of Boost. I think 
> >> that having this API on Windows has an additional benefit. Therefore I 
> >> think it should be implemented as a separate project.
> >

It is, already. Why are we still discussing this?

>  
> > I think I see where you're coming from: you consider the syslog function 
> > a frontend to the syslogd daemon and, from your point of view, a "sink".
> > 
> > This is, on its surface, reasonable, but it ignores the use cases:
> > 
> > 1. How do I write a program that can be configured to send RFC3164 
> > messages to a specific machine?
> > 
> > 2. How do I port a program that uses the syslog function to Windows?
> > 
> > Regarding (1), your proposed solution is for people to require their 
> > users to install a syslogd daemon for Windows. This is unacceptable for 
> > various reasons. A program that can perform its functions from user 
> > space should not install a Windows service, and it would be difficult to 
> > coordinate several such programs to not install conflicting syslogd 
> > services.
> 
> I don't see how including syslog API implementation into Boost.Log can 
> help you with this.

A posix syslog() call can only log to the daemon. Daemon config controls
distrobution from there. Now, if you enforce the syslog "security"
mechanism that will only accept datagrams with a source port of 514, and
you have multiple apps try to bind to 514, you will be in trouble, it
won't work. If instead you allow an arbitrary source port (which is fine
- in reality "restricted" source ports provide no network security) you
can have as many independently configured sinks, each logging to any
destination, local or remote, as you like. 

> You will have to install the syslog daemon somewhere anyway. If not on 
> the machine where your application runs, then on some remote server. And

Assume I already have syslog infrastructure. I don't want to replace it.
I use it to log messages from a whole range of (remote) sources.

>  
> I think that any conflict between several applications using different 
> syslog implementations is not relevant to Boost.Log and at least is 
> solvable as long as there exists a single daemon that conforms to the 
> syslog protocol specification. This daemon can be used for multiple 
> applications.

As you are obviously aware, having already implemented a sink using it,
the syslog API (really, the RFC3164 message format) provides only very
limited message routing/filtering fields. As soon as you rely on it to
route messages, you have lost one of the main features of your proposed
logging library - extensible and flexible message routing/filtering.

>  IOW, I see the following scheme quite viable:
> 
> Application A links with syslog API implementation X.
> 
> Application B links with syslog API implementation Y.
> 

Err - no. Using more than one is not guaranteed to work - unless... See
below.

> Either of the applications A or B may or may not use Boost.Log.
> 

Ok.

> Both X and Y send syslog messages to the daemon Z, which may or may not 
> be part of X or Y distribution. Z may be an independent project at all.
> 

The only publicly defined, portable interface to a syslog daemon is
RFC3164. There is no guarantee that a given
platform/environment/whatever you want to call it uses the same protocol
as another to communicate between a syslog client and a local syslogd.
Thats the "err - no" above.

> There are plenty of Zs already, including the Windows platform. I've 
> found an example of X or Y here:
> 
> http://syslog-win32.sourceforge.net/

Which more or less proves my point that a syslog "client" impl is
trivial, and the portable interface from the client to the daemon is
RFC3164...

> 
> Why would I need to include this functionality into Boost.Log?
> 

So you can combine the power of boost log message routing with the
abilty to route messages to any/many syslog daemons/sinks, be they local
or remote.

> > An alternative might be to implement syslog.lib that does whatever 
> > syslogd does, but from user space. But this duplicates a significant 
> > portion of the Boost.Log goals and functionality; it is effectively a 
> > direct competitor to Boost.Log.
> 

syslogd is always user space.

> I never suggested reimplementing syslog daemon in a user-space library. 
> I agree that this is quite pointless, especially since there are 
> implemented daemons out there.

daemon = long running user space process with no attached console.

> > Regarding (2), it makes perfect sense for Boost.Log to offer a 
> > boost::syslog function as a frontend. This makes porting easy and allows 
> > the program to use Boost.Log's features to centrally filter and route 
> > the log messages.

Nice. I hadn't though about that (I was more interested in a lightweight
ability to log to existing syslog infrastructure). But of course all the
facilities needed for a syslog "daemon" are already in boost log except
for the ability to construct a log message from an received RFC3164
message. 
> 
> Although hypothetically it looks possible to wrap the whole Boost.Log 
> into a library that exposes POSIX syslog API, this would be a quite 
> heavy solution. The syslog API is quite simple and C-oriented, it 
> doesn't need all the bells and whistles that Boost.Log provides.

Doesn't need/doesn't provide - aren't you arguing against your own
objections to implementing RFC3164 logging without going through the
syslog API now?

>  
> Besides, it occurs to me that syslog API is a low-level instrument, and 
> Boost.Log should build on to of it. Imagine for a moment that Pthreads 
> are implemented over Boost.Thread - it looks quite odd to me.
> 

If you built pthreads over boost thread you would restrict its
capabilites somewhat due to restrictions in boost thread. +/- some
syntactic nicities, RAII etc boost thread is a subset of pthreads. If
boost log's ambitions were only to be a C++ "view" of a portable subset
of syslog then your comparison might have some relevance (and the lib
wouldn't get accepted).

> Therefore applications that are explicitly tied with syslog API are 
> better to be ported

What are these apps tied to the syslog API that you are talking about
porting? 

>  with a simple and straightforward implementation of 
> this API, like the one I posted above.

I agree that porting an app that uses syslog can generally be done most
easily by providing the sylog API - that would be an example of Peter
Dimov's (2) above?

The only part of the API required for a sender (1 above) is the part
that builds a message from a few fields passed somewhat arcanely through
the syslog() call and sends the result to UDP port 514. Surely it would
be easier to create such a message directly using the boost log
attribute and formatting system and send it, while at the same time
gaining the ability to route to multiple, local and/or remote, syslog
daemon "sinks"?