[Logging] Support for syslog ?
Dear All, I've decided to port one of my apps to the proposed logging library so that I can write a review. A couple of issues have come up immediately: - This program currently logs using syslog. I was expecting to find a back-end for this logging library for syslog, since it is the standard logging mechanism on POSIX. But I can't see anything. Am I missing something, or is it really not there? - I spent a while trying to work out how to compile the library. I have now come to the conclusion that it is header-only, and that it's only the tests and the examples (called "scenarios") that you might want to compile by following the instructions in the documentation. Is this right? From what I can see so far, using this library is not going to be a compelling alternative to the concise #include <syslog.h> syslog(LOG_WARN,"Blob %d failed to %s",n,x); Cheers, Phil.
Hi Phil,
Dear All,
I've decided to port one of my apps to the proposed logging library so that I can write a review. A couple of issues have come up immediately:
- This program currently logs using syslog. I was expecting to find a back-end for this logging library for syslog, since it is the standard logging mechanism on POSIX. But I can't see anything. Am I missing something, or is it really not there?
At this time I don't have syslog support - but it should be easy to add. I'm a Windows kind of guy, even though I tested my lib on Linux as well.
- I spent a while trying to work out how to compile the library. I have now come to the conclusion that it is header-only, and that it's only the tests and the examples (called "scenarios") that you might want to compile by following the instructions in the documentation. Is this right?
Yes, it is header-only.
From what I can see so far, using this library is not going to be a compelling alternative to the concise
#include <syslog.h> syslog(LOG_WARN,"Blob %d failed to %s",n,x);
Why not? If you say that writing to syslog is that easy, then adding a formater class for syslog should be as easy as what I've just attached. It should work out of the box, but I haven't tested it. Please let me know if it works. I've looked here: http://www.opengroup.org/onlinepubs/009695399/functions/syslog.html Does it handle Unicode (it doesn't seem to)? To add this as a destination, you should call something similar, on your logger: g_l()->writer().add_destination( destination::syslog_no_levels() ); Limitations: - it does NOT care about levels - at this time, it logs everything as LOG_INFO - to care about levels, it's a bit more complicated - I'll post another email in about 10-15 minutes. Best, John -- http://John.Torjo.com -- C++ expert http://blog.torjo.com ... call me only if you want things done right // destination/syslog_no_levels.hpp // Boost Logging library // // Author: John Torjo, www.torjo.com // // Copyright (C) 2007 John Torjo (see www.torjo.com for email) // // Distributed under the Boost Software License, Version 1.0. // (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) // // See http://www.boost.org for updates, documentation, and revision history. // See http://www.torjo.com/log2/ for more details #ifndef JT28092007_destination_syslog_no_levels_HPP_DEFINED #define JT28092007_destination_syslog_no_levels_HPP_DEFINED #if defined(_MSC_VER) && (_MSC_VER >= 1020) # pragma once #endif #include <boost/logging/detail/fwd.hpp> #include <boost/logging/detail/manipulator.hpp> #include <boost/logging/format/destination/convert_destination.hpp> #include <boost/logging/format/destination/file.hpp> #include <syslog.h> namespace boost { namespace logging { namespace destination { /** @brief Writes the string to syslog_no_levels. Note: does not care about levels - always logs as LOG_INFO */ template<class convert_dest = do_convert_destination > struct syslog_no_levels_t : is_generic, boost::logging::op_equal::always_equal { template<class msg_type> void operator()(const msg_type & msg) const { syslog( LOG_INFO, msg.c_str() ); } }; /** @brief syslog_no_levels_t with default values. See syslog_no_levels_t @copydoc syslog_no_levels_t */ typedef syslog_no_levels_t<> syslog_no_levels; }}} #endif
On Thu, Feb 07, 2008 at 11:26:33PM +0200, John Torjo wrote: John, please do not remove "Phil wrote" line.
Hi Phil,
From what I can see so far, using this library is not going to be a compelling alternative to the concise
#include <syslog.h> syslog(LOG_WARN,"Blob %d failed to %s",n,x);
Why not? If you say that writing to syslog is that easy, then adding a formater class for syslog should be as easy as what I've just attached. It should work out of the box, but I haven't tested it. Please let me know if it works.
I've looked here: http://www.opengroup.org/onlinepubs/009695399/functions/syslog.html Does it handle Unicode (it doesn't seem to)?
To be honest I never considered using syslog for application programs. I thought it is only used for system programs, services, ... It is nevertheless very likely that the encoding is just ignored. Write UTF-8 and you will be able to read your stuff in the log. If other programs write KOI8-RU encoded text you will end with a log file with different encoding. That's at least what I guess. Jens
The boost logging library does not seem to support late formatting of strings. From tests that I have carried out we can easily get a couple of orders of magnitude perf improvements by not formatting our strings at runtime, but leaving it to a post processor. This makes the difference between logging being feasible or not in high performance sections of code. Clearly late formatting is difficult because it requires the collation of manifest data of some kind for every log message. My implementation simply logs the boost::format string the first time the log message occurs - but that does mean that it's very awkward to turn logging on and off at run time. An alternative implementation is the Windows WPP library, which runs a preprocessor and puts the results into the symbols files (much more difficult to make cross platform). Have I missed a way to use late formatting with boost::logging? Are there any intentions to introduce late formatting to boost::logging? Thanks Jim ________________________________________________________________________ This e-mail, and any attachment, is confidential. If you have received it in error, do not use or disclose the information in any way, notify me immediately, and please delete it from your system. ________________________________________________________________________
Hi James,
The boost logging library does not seem to support late formatting of strings.
From tests that I have carried out we can easily get a couple of orders of magnitude perf improvements by not formatting our strings at runtime, but leaving it to a post processor. This makes the difference between logging being feasible or not in high performance sections of code.
There are different ways to achieve this. One of them, using boost::logging, is to make a logger: - uses tags - that logs on a dedicated thread. Thus, when you log a message, internally, the message is gathered, tags are gathered as well, and placed into a queue. Then, only on the dedicated thread, the other thread formatting occurs and then the message is written to the destination(s). Example of writing on a dedicated thread: http://torjo.com/log2/doc/html/ded__loger__one__filter_8cpp-example.html
Clearly late formatting is difficult because it requires the collation of manifest data of some kind for every log message. My implementation simply logs the boost::format string the first time the log message occurs - but that does mean that it's very awkward to turn logging on and off at run time.
Why is it awkward? As a side-note : there are many ways to reduce the logging time. One of them is to, when logging, to internally use an optimized string: http://torjo.com/log2/doc/html/namespaceboost_1_1logging_1_1optimize.html Side note 2: to profile the logging time is really easy: http://torjo.com/log2/doc/html/namespaceboost_1_1logging_1_1profile.html
An alternative implementation is the Windows WPP library, which runs a preprocessor and puts the results into the symbols files (much more difficult to make cross platform).
Have I missed a way to use late formatting with boost::logging? Are there any intentions to introduce late formatting to boost::logging?
Note: the boost::logging is extremely flexible. For instance, you can create your own gather_msg class, which uses as many late formatting techniques as you wish. http://torjo.com/log2/doc/html/workflow.html#workflow_2a So for instance you can easily use gather using boost::format, and do the formatting / write to destinations on a dedicated thread. Best, John -- http://John.Torjo.com -- C++ expert http://blog.torjo.com ... call me only if you want things done right
From what I can see so far, using this library is not going to be a compelling alternative to the concise
#include <syslog.h> syslog(LOG_WARN,"Blob %d failed to %s",n,x);
Here you go. Note - in order to work, you need to replace boost/logging/format/destination/convert_destination.hpp - sorry , my bad. test_now.cpp show you how you can easily write to the syslog. Best, John -- http://John.Torjo.com -- C++ expert http://blog.torjo.com ... call me only if you want things done right // convert_destination.hpp // Boost Logging library // // Author: John Torjo, www.torjo.com // // Copyright (C) 2007 John Torjo (see www.torjo.com for email) // // Distributed under the Boost Software License, Version 1.0. // (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) // // See http://www.boost.org for updates, documentation, and revision history. // See http://www.torjo.com/log2/ for more details #ifndef JT28092007_convert_destination_HPP_DEFINED #define JT28092007_convert_destination_HPP_DEFINED #if defined(_MSC_VER) && (_MSC_VER >= 1020) # pragma once #endif #include <boost/logging/detail/fwd.hpp> namespace boost { namespace logging { namespace destination { template<class t> struct into {}; namespace tag { template< class string_ , class param1 , class param2 , class param3 , class param4 , class param5 , class param6 , class param7 , class param8 , class param9 , class param10> struct holder ; } /** @brief Allows writing messages to destinations It has 2 function overloads: - write(message, output) - writes the given message, to the given output - do_convert(message, into<other_type>() ); FIXME */ namespace convert { template<class obj, class char_type, class char_traits> void write(const obj & m, std::basic_ostream<char_type, char_traits> & out) { out << m; } template<class string_, class p1, class p2, class p3, class p4, class p5, class p6, class p7, class p8, class p9, class p10, class char_type, class char_traits> void write(const ::boost::logging::tag::holder<string_,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10> & src, std::basic_ostream<char_type, char_traits> & out ) { typedef typename ::boost::logging::tag::holder<string_,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10>::string_type string_type; out << (const string_type&)src; } template<class char_traits, class char_type> void write(const char_type* m, std::basic_ostream<char_type, char_traits> & out) { out << m; } inline const char_type * do_convert(const char_type * c, const into<const char_type*> &) { return c; } inline const char_type * do_convert(const std::basic_string<char_type> & s, const into<const char_type* > &) { return s.c_str(); } inline const std::basic_string<char_type> & do_convert(const std::basic_string<char_type> & s, const into< std::basic_string<char_type> > &) { return s; } } struct do_convert_destination { template<class msg, class dest> static void write(const msg & m, dest & d) { convert::write(m, d); } template<class msg, class dest> static dest do_convert(const msg & m, const into<dest> &) { return convert::do_convert(m, into<dest>() ); } }; }}} #endif // destination/syslog.hpp // Boost Logging library // // Author: John Torjo, www.torjo.com // // Copyright (C) 2007 John Torjo (see www.torjo.com for email) // // Distributed under the Boost Software License, Version 1.0. // (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) // // See http://www.boost.org for updates, documentation, and revision history. // See http://www.torjo.com/log2/ for more details #ifndef JT28092007_destination_syslog_HPP_DEFINED #define JT28092007_destination_syslog_HPP_DEFINED #if defined(_MSC_VER) && (_MSC_VER >= 1020) # pragma once #endif #include <boost/logging/detail/fwd.hpp> #include <boost/logging/detail/manipulator.hpp> #include <boost/logging/format/destination/convert_destination.hpp> #include <boost/logging/format/destination/file.hpp> #include <boost/logging/detail/level.hpp> #include <syslog.h> namespace boost { namespace logging { namespace destination { /** @brief Writes the string to syslog. Note: does not care about levels - always logs as LOG_INFO */ template<class convert_dest = do_convert_destination > struct syslog_no_levels_t : is_generic, boost::logging::op_equal::always_equal { template<class msg_type> void operator()(const msg_type & msg) const { syslog( LOG_INFO, msg.c_str() ); } }; /** @brief syslog_no_levels_t with default values. See syslog_no_levels_t @copydoc syslog_no_levels_t */ typedef syslog_no_levels_t<> syslog_no_levels; /** @brief Writes the string to syslog. It cares about levels See @ref boost::logging::tag "how to use tags". */ template<class convert = do_convert_format::prepend> struct syslog_t : is_generic, uses_tag< level_t<convert>, ::boost::logging::tag::level >, boost::logging::op_equal::always_equal { typedef convert convert_type; template<class msg_type, class tag_type> void write_tag(msg_type & str, const tag_type & tag) const { syslog( level_to_syslog_level(tag.val) , as_string(str).c_str() ); } private: const hold_string_type& as_string(const hold_string_type & str) { return str; } int level_to_syslog_level( level::type level) { if ( level <= level::debug) return LOG_DEBUG; if ( level <= level::info) return LOG_INFO; if ( level <= level::warning) return LOG_WARN; if ( level <= level::error) return LOG_ERR; if ( level <= level::fatal) return LOG_CRIT; return LOG_EMERG; } }; /** @brief syslog_t with default values. See syslog_t @copydoc syslog_t */ typedef syslog_t<> syslog; }}} #endif #include <boost/logging/format.hpp> #include <boost/logging/format/formatter/tags.hpp> #include <boost/logging/format/formatter/syslog.hpp> namespace bl = boost::logging; typedef bl::tag::holder<bl::default_, bl::tag::level> tag_holder; BOOST_LOG_FORMAT_MSG(tag_holder) BOOST_LOG_DESTINATION_MSG(tag_holder) typedef bl::logger_format_write< > logger_type; #define L_(lvl) BOOST_LOG_USE_LOG_IF_LEVEL(g_l(), g_log_filter(), lvl ) .set_tag( BOOST_LOG_TAG_LEVEL(lvl) ) BOOST_DEFINE_LOG_FILTER(g_log_filter, bl::level::holder ) BOOST_DEFINE_LOG(g_l, logger_type) int main() { g_l()->writer().add_formatter( bl::formatter::idx(), "[%] " ); g_l()->writer().add_formatter( bl::formatter::append_newline() ); // write to cout and syslog g_l()->writer().add_destination( bl::destination::cout() ); g_l()->writer().add_destination( bl::destination::syslog() ); g_l()->mark_as_initialized(); int i = 1; L_(debug) << "this is so cool " << i++; L_(error) << "first error " << i++; }
From what I can see so far, using this library is not going to be a compelling alternative to the concise
#include <syslog.h> syslog(LOG_WARN,"Blob %d failed to %s",n,x);
Yeah, we used to do that. Once you've had a few crashes due to passing the wrong argument types in the variable argument list, that _only_ happen when you hit the error condition that that line is supposed to help you diagnose, Boost.Logging looks very compelling indeed... :) Colin
Colin Caughie wrote:
From what I can see so far, using this library is not going to be a compelling alternative to the concise
#include <syslog.h> syslog(LOG_WARN,"Blob %d failed to %s",n,x);
Yeah, we used to do that. Once you've had a few crashes due to passing the wrong argument types in the variable argument list, that _only_ happen when you hit the error condition that that line is supposed to help you diagnose, Boost.Logging looks very compelling indeed... :)
I read that as an argument for more thorough unit-testing, not for switching to a different tool. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
Colin Caughie wrote:
From what I can see so far, using this library is not going to be a compelling alternative to the concise
#include <syslog.h> syslog(LOG_WARN,"Blob %d failed to %s",n,x);
Yeah, we used to do that. Once you've had a few crashes due to passing the wrong argument types in the variable argument list, that _only_ happen when you hit the error condition that that line is supposed to help you diagnose, Boost.Logging looks very compelling indeed... :)
You need a compiler that type-checks printf-like function calls: in /usr/include/sys/syslog.h: extern void syslog (int __pri, __const char *__fmt, ...) __attribute__ ((__format__(__printf__, 2, 3))); test.c: #include <syslog.h> void f(char* s) { syslog(LOG_INFO,"Foo %d",s); } $ gcc -W -Wall -c /tmp/test.c /tmp/test.c: In function ‘f’: /tmp/test.c:3: warning: format ‘%d’ expects type ‘int’, but argument 3 has type ‘char *’ Phil.
participants (6)
-
Colin Caughie -
James Talbut -
Jens Seidel -
John Torjo -
Phil Endecott -
Stefan Seefeld