The syslog-ng 3.0 Administrator Guide

All messages generated internally by syslog-ng use this special source. To collect warnings, errors and notices from syslog-ng itself, include this source in one of your source statements.

internal()

The syslog-ng application will issue a warning upon startup if none of the defined log paths reference this driver.

	Example 3.7. Using the internal() driver
	source s_local { internal(); };

3.3.1.1. Log statistics

Periodically, syslog-ng sends a message containing statistics about the received messages, and about any lost messages since the last such message. It includes a processed entry for every source and destination, listing the number of messages received or sent, and a dropped entry including the IP address of the server for every destination where syslog-ng has lost messages. The center(received) entry shows the total number of messages received from every configured sources.

The following is a sample log statistics message for a configuration that has a single source (s_local) and a network and a local file destination (d_network and d_local, respectively). All incoming messages are sent to both destinations.

Log statistics;
                    dropped='tcp(AF_INET(192.168.10.1:514))=6439',
                    processed='center(received)=234413',
                    processed='destination(d_tcp)=234413',
                    processed='destination(d_local)=234413',
                    processed='source(s_local)=234413'

Log statistics can be also retrieved on-demand using the echo STATS | nc -U var/run/syslog-ng.ctl command. This returns a list of source groups and destinations, as well as the number of processed messages for each. The verbosity of the statistics can be set using the stats_level() option. See Section 8.9, “Global options” for details.

	Note
	To query the statistics, you need the OpenBSD-style netcat application. The netcat included in most Linux distributions is a GNU-style version that is not suitable to query the statistics of syslog-ng. An alternative is to use the socat application: echo STATS | socat -vv UNIX-CONNECT:/opt/syslog-ng/var/run/syslog-ng.ctl -.

Collects log messages from plain-text files, e.g., from the logfiles of an Apache webserver.

The syslog-ng application notices if a file is renamed or replaced with a new file, so it can correctly follow the file even if logrotation is used. When syslog-ng is restarted, it records the position of the last sent log message, and continues to send messages from this position after the restart.

The file driver has a single required parameter specifying the file to open. For the list of available optional parameters, see Section 8.1.2, “file()”.

Declaration:
                file(filename);

In syslog-ng PE, the filename (but not the pathname) may include wildcard characters (e.g., *). Note that when using wildcards in filenames, always set how often syslog-ng should check the file for new messages using the follow_freq() parameter.

When using wildcards, syslog-ng PE monitors every matching file, and can receive new log messages from any of the files. However, monitoring (polling) many files (i.e., more than ten) has a significant overhead and may affect performance. On Linux this overhead is not so significant, because syslog-ng PE uses the inotify feature of the kernel.

	Example 3.8. Using the file() driver
	source s_file { file("/var/log/messages"); };

	Example 3.9. Using wildcards in the filename
	The following example monitors every file with the .log extension in the /var/application directory for log messages. Note that only syslog-ng PE supports wildcards in the file and pathnames. source s_file { file("/var/application/*.log" follow_freq(1));};

	Example 3.10. Monitoring multiple directories
	The following example reads files having the .log extension from the /var/application/ directory and its subdirectories. Note that only syslog-ng PE supports recursive directory handling and wildcards in the file and pathnames. source s_file_subdirectories { file("/var/application/*.log" recursive(yes) follow_freq(1) log_fetch_limit(100) );};

The kernel usually sends log messages to a special file (/dev/kmsg on BSDs, /proc/kmsg on Linux). The file() driver reads log messages from such files. The syslog-ng application can periodically check the file for new log messages if the follow_freq() option is set.

Note

On Linux, the klogd daemon can be used in addition to syslog-ng to read kernel messages and forward them to syslog-ng. klogd used to preprocess kernel messages to resolve symbols etc., but as this is deprecated by ksymoops there is really no point in running both klogd and syslog-ng in parallel. Also note that running two processes reading /proc/kmsg at the same time might result in dead-locks. When using syslog-ng to read messages from the /proc/kmsg file, syslog-ng automatically disables the follow_freq() parameter to avoid blocking the file.

The pipe driver opens a named pipe with the specified name and listens for messages. It is used as the native message delivery protocol on HP-UX.

The pipe driver has a single required parameter, specifying the filename of the pipe to open. For the list of available optional parameters, see Section 8.1.3, “pipe()”.

Declaration:
                pipe(filename);

	Note
	As of syslog-ng Open Source Edition 3.0.2, pipes are created automatically. In earlier versions, you had to create the pipe using the mkfifo(1) command.

Pipe is very similar to the file() driver, but there are a few differences, for example pipe() opens its argument in read-write mode, therefore it is not recommended to be used on special files like /proc/kmsg.

	Warning
	It is not recommended to use pipe() on anything else than real pipes.

	Example 3.11. Using the pipe() driver
	source s_pipe { pipe("/dev/pipe" pad_size(2048)); };

Solaris uses its STREAMS framework to send messages to the syslogd process. Solaris 2.5.1 and above uses an IPC called door in addition to STREAMS, to confirm the delivery of a message. The syslog-ng application supports the IPC mechanism via the door() option (see below).

	Note
	The sun-streams() driver must be enabled when the syslog-ng application is compiled (see ./configure --help). The pre-compiled binaries of syslog-ng Open Source Edition purchased from BalaBit and syslog-ng Premium Edition have this option enabled.

The sun-streams() driver has a single required argument specifying the STREAMS device to open, and the door() option. For the list of available optional parameters, see Section 8.1.5, “sun-streams() driver”.

Declaration:
        sun-streams(name_of_the_streams_device door(filename_of_the_door));

	Example 3.12. Using the sun-streams() driver
	source s_stream { sun-streams("/dev/log" door("/etc/.syslog_door")); };

The syslog() driver enables to receive messages from the network using the new standard syslog protocol and message format (also called IETF-syslog protocol; described in RFC 5424-28, see Section 2.19.2, “IETF-syslog messages”). UDP, TCP, and TLS-encrypted TCP can all be used to transport the messages.

For the list of available optional parameters, see Section 8.1.6, “syslog()”.

Declaration:
            syslog(ip() port() transport() options());

Example 3.13. Using the syslog() driver

TCP source listening on the localhost on port 1999. source s_syslog { syslog(ip(127.0.0.1) port(1999) transport("tcp")); }; UDP source with defaults. source s_udp { syslog( transport("udp")); }; Encrypted source where the client is also authenticated. See Section 8.10, “TLS options” for details on the encryption settings. source s_syslog_tls{ syslog( ip(10.100.20.40) transport("tls") tls( peer-verify(required-trusted) ca_dir('/opt/syslog-ng/etc/syslog-ng/keys/ca.d/') key_file('/opt/syslog-ng/etc/syslog-ng/keys/server_privatekey.pem') cert_file('/opt/syslog-ng/etc/syslog-ng/keys/server_certificate.pem') ) );};

The tcp(), tcp6(), udp(), udp6() drivers can receive messages from the network using the TCP and UDP networking protocols. The tcp6() and udp6() drivers use the IPv6 network protocol, while tcp() and udp() use IPv4.

UDP is a simple datagram oriented protocol, which provides "best effort service" to transfer messages between hosts. It may lose messages, and no attempt is made at the protocol level to retransmit such lost messages. The BSD-syslog protocol traditionally uses UDP.

TCP provides connection-oriented service, which basically means that the path of the messages is flow-controlled. Along this path, each message is acknowledged, and retransmission is done for lost packets. Generally it is safer to use TCP, because lost connections can be detected, and no messages get lost, assuming that the TCP connection does not break. When a TCP connection is broken the 'in-transit' messages that were sent by syslog-ng but not yet received on the other side are lost. (Basically these messages are still sitting in the socket buffer of the sending host and syslog-ng has no information about the fate of these messages).

The tcp() and udp() drivers do not have any required parameters. By default they bind to the 0.0.0.0:514 address, which means that syslog-ng will listen on all available interfaces, port 514. To limit accepted connections to only one interface, use the localip() parameter as described below. For the list of available optional parameters, see Section 8.1.7, “tcp(), tcp6(), udp() and udp6()”.

Declaration:
                tcp([options]);
                udp([options]);

	Note
	The tcp port 514 is reserved for use with rshell, so select a different port if syslog-ng and rshell is used at the same time.

If you specify a multicast bind address to udp() and udp6(), syslog-ng will automatically join the necessary multicast group. TCP does not support multicasting.

The syslog-ng Premium Edition application supports TLS (Transport Layer Security, also known as SSL) for the tcp() and tcp6() drivers. See the TLS-specific options below and Section 3.13, “Encrypting log messages with TLS” for details. For the list of available optional parameters, see Section 8.1.5, “sun-streams() driver”.

Example 3.14. Using the udp() and tcp() drivers

A simple udp() source with default settings. source s_udp { udp(); };# An UDP source with default settings. A TCP source listening on the localhost interface, with a limited number of connections allowed. source s_tcp { tcp(ip(127.0.0.1) port(1999) max-connections(10)); }; A TCP source listening on a TLS-encrypted channel. source s_tcp { tcp(ip(127.0.0.1) port(1999) tls(peer-verify('required-trusted') key_file('/opt/syslog-ng/etc/syslog-ng/syslog-ng.key') cert_file('/opt/syslog-ng/etc/syslog-ng/syslog-ng.crt'))); }; A TCP source listening for messages using the IETF-syslog message format. Note that for transferring IETF-syslog messages, generally you are recommended to use the syslog() driver on both the client and the server, as it uses both the IETF-syslog message format and the protocol. See Section 3.3.5, “Collecting messages using the IETF syslog protocol” for details. source s_tcp_syslog { tcp(ip(127.0.0.1) port(1999) flags(syslog-protocol) ); };

The unix-stream() and unix-dgram() drivers open an AF_UNIX socket and start listening on it for messages. The unix-stream() driver is primarily used on Linux and uses SOCK_STREAM semantics (connection oriented, no messages are lost); while unix-dgram() is used on BSDs and uses SOCK_DGRAM semantics: this may result in lost local messages if the system is overloaded.

To avoid denial of service attacks when using connection-oriented protocols, the number of simultaneously accepted connections should be limited. This can be achieved using the max-connections() parameter. The default value of this parameter is quite strict, you might have to increase it on a busy system.

Both unix-stream and unix-dgram have a single required argument that specifies the filename of the socket to create. For the list of available optional parameters, see Section 8.1.8, “unix-stream() and unix-dgram()”

Declaration: 
                unix-stream(filename [options]);
                unix-dgram(filename [options]);

	Note
	syslogd on Linux originally used SOCK_STREAM sockets, but some distributions switched to SOCK_DGRAM around 1999 to fix a possible DoS problem. On Linux you can choose to use whichever driver you like as syslog clients automatically detect the socket type being used.

The difference between the unix-stream and unix-dgram drivers is similar to the difference between the TCP and UDP network protocols. Use the following guidelines to select which driver to use in a particular situation:

Choose unix-stream if you would choose TCP (stream) instead of UDP (datagram). The unix-stream driver offers the following features:

Choose unix-dgram if you would choose TCP (stream) over UDP (datagram). The unix-dgram driver offers the following features:

However, the client does not notice if a message is lost when using the unix-dgram driver.

	Example 3.15. Using the unix-stream() and unix-dgram() drivers
	source s_stream { unix-stream("/dev/log" max-connections(10)); }; source s_dgram { unix-dgram("/var/run/log"); };