3.3.2. Collecting messages from text files

Collects log messages from plain-text files, e.g., from the logfiles of an Apache webserver.

The syslog-ng application notices if a file is renamed or replaced with a new file, so it can correctly follow the file even if logrotation is used. When syslog-ng is restarted, it records the position of the last sent log message, and continues to send messages from this position after the restart.

The file driver has a single required parameter specifying the file to open. For the list of available optional parameters, see Section 8.1.2, “file()”.

Declaration:
                file(filename);

In syslog-ng PE, the filename (but not the pathname) may include wildcard characters (e.g., *). Note that when using wildcards in filenames, always set how often syslog-ng should check the file for new messages using the follow_freq() parameter.

When using wildcards, syslog-ng PE monitors every matching file, and can receive new log messages from any of the files. However, monitoring (polling) many files (i.e., more than ten) has a significant overhead and may affect performance. On Linux this overhead is not so significant, because syslog-ng PE uses the inotify feature of the kernel.

[Example] Example 3.8. Using the file() driver
source s_file { file("/var/log/messages"); };
[Example] Example 3.9. Using wildcards in the filename

The following example monitors every file with the .log extension in the /var/application directory for log messages. Note that only syslog-ng PE supports wildcards in the file and pathnames.

source s_file { file("/var/application/*.log" follow_freq(1));};
[Example] Example 3.10. Monitoring multiple directories

The following example reads files having the .log extension from the /var/application/ directory and its subdirectories. Note that only syslog-ng PE supports recursive directory handling and wildcards in the file and pathnames.

source s_file_subdirectories { file("/var/application/*.log"
                    recursive(yes) 
                    follow_freq(1) 
                    log_fetch_limit(100)
                    );};

The kernel usually sends log messages to a special file (/dev/kmsg on BSDs, /proc/kmsg on Linux). The file() driver reads log messages from such files. The syslog-ng application can periodically check the file for new log messages if the follow_freq() option is set.

[Note] Note

On Linux, the klogd daemon can be used in addition to syslog-ng to read kernel messages and forward them to syslog-ng. klogd used to preprocess kernel messages to resolve symbols etc., but as this is deprecated by ksymoops there is really no point in running both klogd and syslog-ng in parallel. Also note that running two processes reading /proc/kmsg at the same time might result in dead-locks.

When using syslog-ng to read messages from the /proc/kmsg file, syslog-ng automatically disables the follow_freq() parameter to avoid blocking the file.


© 2007-2010 BalaBit IT Security
Please send your comments or documentation bugs to: documentation@balabit.com