3.2.2 Sat, 15 Jan 2011 13:50:35 +0100 Bugfixes: * Fixed a possible segmentation fault when the port number is changed for a TCP source, the configuration is reloaded and there were open connections for the old port, which send messages after the SIGHUP. This behaviour has been broken since 3.0.1. * Fixed a possible security issue on FreeBSD and on platforms where mode_t is an unsigned 16 bit value. On these platforms syslog-ng may be using 0xFFFF as the permission bits causing log files to be world readable/writable/executable/setuid. * Fixed leaking the contents of internal() messages (such as MARK or the statistics message). * Fixed current time tracking when calculating the time in the correllation engine. * When the patterndb file got reloaded the correllation state was dropped. This behaviour was fixed. * Really ignore invalid persist-state files, which caused syslog-ng startup to fail previously. * Added the missing support for blocks inside log {} statements. * Fixed a configuration init error when the same db-parser() instance is referenced from multiple log paths. * Fixed handling the port() options for SQL destinations. Platform specific: * Added cygwin support to the system() source. Build related: * syslog-ng modules are now linked with "-module -no-undefined" parameters, pdbtool and unit tests are using -dlpreopen when explicitly linking against such modules. * The core patterndb functionality got split off to a separate library installed to $libdir to make it easier to be used by pdbtool. * Fixed support for an explicit --exec-prefix configure parameter. Earlier if exec_prefix was different from prefix, the installation layout produced unworkable binaries. * If no OpenSSL libraries are available, pdbtool patternize still can work, although in this case proper UUID generation is not possible. * If syslog-ng is compiled against an old glib (earlier than 2.13), it'll not use an API that is present in newer ones. Please note however that there might be other similar compatibility issues with old Glib versions. * Updated cygwin packaging files. * Don't use -wno-pointer-sign in dbparser if gcc doesn't support it. This will emit a lot of warnings, but still make the code possible to compile with older gcc versions. Credits: syslog-ng is developed as a community project. All changes and improvements requires effort, and this effort is really appreciated. Writing code, testing changes or simply providing use-cases and information on one's setup will make syslog-ng better. Here are the people, listed in no specific order who made this release possible. * Balázs Németh (BalaBit) * Sándor Gellér (Morgan Stanley) * Péter Czanik (BalaBit) * Owen Mann (Interactive Data) * Zhengxiang Pan (Alcatel Lucent) * Corinna Vinschen (RedHat) * Eric Berggren (Apple) * Gergely Nagy (BalaBit) * Matthew Hall * LEBRETON Philippe * Craig Bell * Steven Chamberlain * Balázs Scheidler (BalaBit) Thanks for their efforts, it is appreciated. 3.2.1 Tue, 23 Nov 2010 08:59:47 +0100 This is the first release of the new major version of syslog-ng, dubbed 3.2. There are far-reaching changes in this release, the summary of the new features is the longest list ever since the first syslog-ng release in 1998. db-parser(): ============ * Added support for message correllation in db-parser. See the relevant blog posts for more information: http://bazsi.blogs.balabit.com/2010/10/syslog-ng-correllation-updated/ http://bazsi.blogs.balabit.com/2010/09/syslog-ng-correllation/ * Added "pdbtool patternize", which implements automatic patterndb generation from a sample log file. http://gyp.blogs.balabit.com/2010/01/introducing-pdbtool-patternize/ * Added pdbtool validation support, using the "pdbtool test --validate". Requires an installed xmllint program. * pdbtool is now able to merge patterndb XML files recursively in order to make it easy to use the results of the patterndb project. * db-parser() automatically assigns class-specific tags to messages, this means that a message classified "system" will get a ".classifier.system" tag in addition to storing the class in a name-value pair named ${.classifier.class} * It is now possible to use multiple program name patterns for a single ruleset in patterndb. * pdbtool match is now able to read a file containing syslog messages and apply patterndb and a filter expression on the contents. http://bazsi.blogs.balabit.com/2010/07/patterndb-grep-on-steroids.html * pdbtool test is now able to perform pattern testing automatically based on the supplied example log message. http://marci.blogs.balabit.com/2010/07/pdbtool-test-and-pattern-database.html Message model: ============== * Added template functions framework and some initial functions: http://bazsi.blogs.balabit.com/2010/09/introducing-template-functions/ The new functions are: $(echo), $(grep) and $(if) * Added support for comparison operators in filter expressions, e.g. it is now possible to use "$FACILITY_NUM" < "5". String and numeric operators are also provided, the same way as in perl. * Added $(ipv4-to-int) template function to convert an IP address to its numeric representation. * It is now possible to supply a filter to rewrite expressions and only apply the rewrite rule in case the filter matches. https://lists.balabit.hu/pipermail/syslog-ng/2010-July/014565.html Architecture: ============= * Plugins: the new architecture replaces the old monolithic one, all syslog-ng functionality is loaded from external plugins when needed. It is possible to write plugins to extend syslog-ng functionality in the following areas: - sources - destinations - filter expression - parsers - rewrite ops - message format http://bazsi.blogs.balabit.com/2010/04/syslog-ng-32-changes.html http://bazsi.blogs.balabit.com/2010/07/syslog-ng-contributions-redefined.html * The framework for a "syslog-ng configuration library" (aka SCL) a collection of configuration snippets installed along syslog-ng, simplifying the authoring of syslog-ng configuration files. http://git.balabit.hu/?p=bazsi/syslog-ng-3.2.git;a=commit;h=287993339599deac0442e26355c600b5aee63583 http://bazsi.blogs.balabit.com/2010/07/syslog-ng-contributions-redefined.html * Support for reusable configuration snippets, similar to macros with parameters, named "blocks". http://bazsi.blogs.balabit.com/2010/04/syslog-ng-32-opened-experimental-blocks.html * Added a confgen plugin that includes the output of a program into the configuration file, making it possible to generate configuration file snippets dynamically. http://git.balabit.hu/?p=bazsi/syslog-ng-3.2.git;a=commit;h=5248ef6c49ff3af0b3c896448360073606c9c7d7 Compatibility: ============== * Added support to process native syslog.conf file using the syslogconf SCL plugin. http://bazsi.blogs.balabit.com/2010/09/syslog-ng-now-supports-the-syslog-conf-file-format/ * syslog-ng now automatically detects if an incoming message is in RFC3164 or RFC5424 format. This means that the syslog driver can be used to process both. New sources: * Support for BSD-style process accounting logs via the pacct() source driver defined in by SCL and the underlying pacctformat plugin. http://bazsi.blogs.balabit.com/2010/07/syslog-ng-and-process-accounting.html SQL driver enhancements: ======================== * Support for explicit COMMITs in the SQL driver, this speeds up SQL INSERT rate significantly if flush_lines() is non-zero. http://bazsi.blogs.balabit.com/2010/04/explicit-transaction-support-in-sql.html Other changes: ============== * Persistent state containing the current file position for file sources is now continously updated during runtime, instead of updating it only at exit, which makes it much more reliable in case syslog-ng doesn't terminate normally. * Better syntax error reporting in the configuration file. http://bazsi.blogs.balabit.com/2010/04/syslog-ng-32-changes.html * It is now possible to use multiple parser expressions in a single parser object, similar to rewrite rules. * Added support for using the include statement from anywhere in the configuration file, instead of only at top-level. Also introduced syslog-ng "global values" that can be defined and the substituted anywhere in the configuration file. http://git.balabit.hu/?p=bazsi/syslog-ng-3.2.git;a=commit;h=1203267c465256c99e622edf11e226301170f1c7 http://git.balabit.hu/?p=bazsi/syslog-ng-3.2.git;a=commit;h=52098762f27cde059e8b8ecda67691df85364e6d * Default configuration file supplied as part of SCL. Incompatible changes: ===================== * syslog-ng traditionally expected an optional hostname field even when a syslog message is received on a local transport (e.g. /dev/log). However no UNIX version is known to include this field. This caused problems when the application creating the log message has a space in its program name field. This behaviour has been changed for the unix-stream/unix-dgram/pipe drivers if the config version is 3.2 and can be restored by using an explicit 'expect-hostname' flag for the specific source. Compared to 3.2beta1: ===================== Bugfixes: * Fixed Linux capability support for unix-stream() and file() destinations (Zbigniew Krzystolik) * Fixed segmentation faults in "pdbtool match" reported by Peter Czanik. * Fixed pdbtool match --debug-pattern to correctly display & colorize its output. * Fixed negated tags() filtering. * The hostname wasn't always properly NUL terminated, causing binary garbage to get into the logs in case chain_hostnames() option is used. * Fixed signed/unsigned comparison problem in db-parser() pattern matching, possibly causing the db-parser() to mismatch on utf8 data. * The db-parser() correllation state is kept accross SIGHUPs. Other changes: * Added man pages for loggen, syslog-ng-ctl. Updated man pages for all other commands. (Robert Fekete) * Removed the requirement to use UUIDs in patterndb files. * The Debian packaging built into the source now builds a pluginised syslog-ng binary correctly. * The correllation engine now also follows system time to cause pending events to time out even if there's no incoming log traffic. * When using "pdbtool match" with correllation, pending events accumulated until the end of the file are all run automatically. * Added patterndb v4 XML schema. 3.2beta1 Mon, 11 Oct 2010 12:25:07 +0200 Changes and new features destined to the syslog-ng 3.2 release are complete, and starting with this release, only bugfixes and minor changes are possible. There's only one exception to this: the correllation framework in db-parser() is still considered experimental and is recommended for early adopters only. This beta has gone through some testing and initial blocker problems were fixed before the release. Right now I'm not aware of any serious issues, but as always, testing is appreciated. New features since 3.2alpha2: * Added support for message correllation in db-parser. See the relevant blog posts for more information: http://bazsi.blogs.balabit.com/2010/10/syslog-ng-correllation-updated/ http://bazsi.blogs.balabit.com/2010/09/syslog-ng-correllation/ * Added "pdbtool patternize", which implements automatic patterndb generation from a sample log file. http://gyp.blogs.balabit.com/2010/01/introducing-pdbtool-patternize/ * Added template functions framework and some initial functions: http://bazsi.blogs.balabit.com/2010/09/introducing-template-functions/ The new functions are: $(echo), $(grep) and $(if) * Added support to process native syslog.conf file using the syslogconf SCL plugin. http://bazsi.blogs.balabit.com/2010/09/syslog-ng-now-supports-the-syslog-conf-file-format/ * Added support for comparison operators in filter expressions, e.g. it is now possible to use "$FACILITY_NUM" < "5". String and numeric operators are also provided, the same way as in perl. * syslog-ng now automatically detects if an incoming message is in RFC3164 or RFC5424 format. This means that the syslog driver can be used to process both. * Added pdbtool validation support, using the "pdbtool test --validate". Requires an installed xmllint program. * pdbtool is now able to merge patterndb XML files recursively in order to make it easy to use the results of the patterndb project. * db-parser() automatically assigns class-specific tags to messages, this means that a message classified "system" will get a ".classifier.system" tag in addition to storing the class in a name-value pair named ${.classifier.class} * It is now possible to use multiple program name patterns for a single ruleset in patterndb. * Added $(ipv4-to-int) template function to convert an IP address to its numeric representation. Bugfixes since 3.2alpha2: * Fixed a possible infinite loop in "pdbtool test" in case program/message was missing from the sample message. * SQL: revert don't require the current CVS version of libdbi * Don't report "this config file version is too old" multiple times. * Underscore and dash are assumed to be equivalent in plugin names. * Various memory leaks were plugged. Removed functions: * Removed the use_time_recvd() global and per-destination option, deprecated since 3.0. Can be substituted with $R_ prefix in macro names. Other changes: * Restructured the source tree in order to make compilations of independent plugins easier and faster. Modules go to modules/ subdirectory, the core lives under lib/ and the main executables go into syslog-ng/ * SCL paths are determined relative to ${datadir} instead of ${prefix} to make distribution packaging easier. * Pass -avoid-version when linking modules. * syslog-ng now requires bison 2.4, this is also checked by the configure script.