9.2. Destination drivers

Destination drivers output log messages to somewhere outside syslog-ng e.g., to a file or a network socket.

9.2.1. Options common for every destination

Some of the parameters affecting message formatting and sending are common for almost all destinations. The following options are available for every destination, except for sql() and usertty().

Name Type Default Description
flags()   empty set
flush_lines() number Use global setting. Specifies how many lines are flushed to a destination at a time. Syslog-ng waits for this number of lines to accumulate and sends them off in a single batch. Setting this number high increases throughput as fully filled frames are sent to the network, but also increases message latency. The latency can be limited by the use of the flush_timeout option.
flush_timeout() time in milliseconds Use global setting. Specifies the time syslog-ng waits for lines to accumulate in its output buffer. See the flush_lines option for more information.
frac_digits() number 0 The syslog-ng application can store fractions of a second in the timestamps. The frac_digits() parameter specifies the number of digits stored. The digits storing the fractions are padded by zeros if the original timestamp of the message specifies only seconds. Fractions can always be stored for the time the message was received.
fsync() yes or no no Forces an fsync() call on the destination fd after each write. Note: this may seriously degrade performance.
log_fifo_size() number Use global setting. The number of entries in the output buffer (output fifo).
suppress() seconds 0 (disabled) When enabled and syslog-ng would send several messages with identical content to the destination, syslog-ng sends only a single message and a line Last message repeated n times.. The parameter of this option specifies the number of seconds syslog-ng waits for identical messages.
sync_freq() number Use global setting. This setting is an obsolete alias of the flush_lines() option.
template() string A format conforming to the default logfile format. Specifies a template defining the logformat to be used in the destination. Macros are described in Section 9.5, “Macros”. Please note that for network destinations it might not be appropriate to change the template as it changes the on-wire format of the syslog protocol which might not be tolerated by stock syslog receivers (like syslogd or syslog-ng itself). For network destinations make sure the receiver can cope with the custom format defined.
template_escape() yes or no yes Turns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server.
throttle() number 0 Sets the maximum number of messages sent to the destination per second. Use this output-rate-limiting functionality only when using disk-buffer as well to avoid the risk of losing messages. Specifying 0 or a lower value sets the output limit to unlimited.
timezone() timezone offset in seconds unspecified Convert timestamps to the timezone specified by this option. If this option is not set then the original timezone information in the message is used.
ts_format() rfc3164, bsd, rfc3339, iso rfc3164 Override the global timestamp format (set in the global ts_format() parameter) for the specific destination. See also Section 7.5, “A note on timezones and timestamps”.

Table 9.5. Common options for destination drivers


[Note] Note

The usertty destination does not support templates.

9.2.2. file()

The file driver is one of the most important destination drivers in syslog-ng. It allows to output messages to the specified file, or to a set of files.

The destination filename may include macros which get expanded when the message is written, thus a simple file() driver may crete several files. For more information on available macros see Section 9.5, “Macros”.

If the expanded filename refers to a directory which does not exist, it will be created depending on the create_dirs() setting (both global and a per destination option).

[Note] Note

When using the file() destination, update the configuration of your log rotation program to rotate these files. Otherwise, the log files can become very large.

[Warning] Warning

Since the state of each created file must be tracked by syslog-ng, it consumes some memory for each file. If no new messages are written to a file within 60 seconds (controlled by the time_reap() global option), it is closed, and its state is freed.

Exploiting this, a DoS attack can be mounted against the system. If the number of possible destination files and its needed memory is more than the amount available on the syslog-ng server.

The most suspicious macro is $PROGRAM, where the number of possible variations is rather high. Do not use the $PROGRAM macro in insecure environments.

Apart from the common destination options described in Table 9.5, “Common options for destination drivers” the file() destination has the following options:

Name Type Default Description
create_dirs() yes or no no Enable creating non-existing directories.
dir_group() string root The group of directories created by syslog-ng.
dir_owner() string root The owner of directories created by syslog-ng.
dir_perm() number 0600 The permission mask of directories created by syslog-ng. Log directories are only created if a file after macro expansion refers to a non-existing directory, and directory creation is enabled (see the create_dirs() option below). For octal numbers prefix the number with 0, e.g., use 0755 for rwxr-xr-x.
group() string root Set the group of the created file to the one specified.
overwrite_if_older() number 0 If set to a value higher than 0, before writing to a file, syslog-ng checks whether this file is older than the specified amount of time (specified in seconds). If so, it removes the existing file and the line to be written is the first line of a new file having the same name. In combination with e.g.: the $WEEKDAY macro, this can be used for simple log rotation, in case not all history has to be kept.
owner() string root Set the owner of the created file to the one specified.
perm() number 0600 The permission mask of the file if it is created by syslog-ng. For octal numbers prefix the number with 0, e.g., use 0755 for rwxr-xr-x.
remove_if_older() number 0 Obsolete alias of the overwrite_if_older() option.

Table 9.6. Options for file()


[Example] Example 9.8. Using the file() driver
destination d_file { file("/var/log/messages" ); };
[Example] Example 9.9. Using the file() driver with macros in the file name and a template for the message
destination d_file {
        file("/var/log/$YEAR.$MONTH.$DAY/messages"
             template("$HOUR:$MIN:$SEC $TZ $HOST [$LEVEL] $MSG $MSG\n")
             template_escape(no));
};

9.2.3. pipe()

This driver sends messages to a named pipe like /dev/xconsole.

The pipe driver has a single required parameter, specifying the filename of the pipe to open.

Declaration:
pipe(filename);
[Note] Note

You have to create this pipe using mkfifo(1).

Apart from the common destination options described in Table 9.5, “Common options for destination drivers” the pipe() destination has the following options:

Name Type Default Description
group() string root Set the group of the pipe to the one specified.
owner() string root Set the owner of the pipe to the one specified.
perm() number 0600 The permission mask of the pipe. For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.
template() string A format conforming to the default logfile format. Specifies a template which defines the logformat to be used. Possible macros are the same as for the file() destination.
template_escape() yes or no yes Turns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server.

Table 9.7. Options for pipe()


[Example] Example 9.10. Using the pipe() driver
destination d_pipe { pipe("/dev/xconsole"); };

9.2.4. program()

This driver executes the specified program with the specified arguments and sends messages to the standard input (stdin) of the child.

The program() driver has a single required parameter, specifying a program name to start. The program is executed with the help of the current shell, so the command may include both file patterns and I/O redirection, they will be processed.

Declaration: 
program(command_to_run);
[Note] Note

Version 1.6 of syslog-ng executed the program once at startup, and kept it running until SIGHUP or exit. The reason was to prevent starting up a large number of programs for messages, which would have enabled an easy DoS attack.

Versions 2.0 and later restart the program if it exits for reliability reasons. However it is not recommended to launch programs for single messages as that might easily cause a DoS for the system.

The program destination supports all common destination options described in Table 9.5, “Common options for destination drivers”.

Note that the message format does not include the priority and facility values by default (these values were automatically added in version 1.6.x of syslog-ng). To add these values, specify a template for the program destination, as shown in the following example.

[Example] Example 9.11. Using the program() destination driver
destination d_prog { program("/bin/script" template("<$PRI>$DATE $HOST $MSG\n"); };

9.2.5. sql()

This driver sends messages into an SQL database. Inserting the records into the database is performed by a separate thread. The syslog-ng application automatically performs the escaping required to insert the messages into the database. The table and value parameters can include macros (see Section 9.5, “Macros” for details).

[Note] Note

In order to use the sql() destination, syslog-ng Premium Edition must run in server mode. Typically, only the central syslog-ng Premium Edition server uses the sql() destination.

Declaration: 
sql(database_type host_parameters database_parameters [options]);
[Note] Note

In addition to the standard syslog-ng packages, the sql() destination requires database-specific packages to be installed. Refer to the section appropriate for your platform in Chapter 4, Installing syslog-ng.

The sql() driver is currently not available for every platform that is supported by syslog-ng. For a list of platforms that support the sql() driver, visit http://www.balabit.com/network-security/syslog-ng/central-syslog-server/.

The common destination options described in Table 9.5, “Common options for destination drivers” are not available for the sql() destination. The sql() destination has the following options:

Name Type Default Description
columns string list "date", "facility", "level", "host", "program", "pid", "message" Name of the columns storing the data in fieldname [dbtype] format. The [dbtype] parameter is optional, and specifies the type of the field.
database string n/a Name of the database that stores the logs.
host hostname or IP address n/a Hostname of the database server. Note that Oracle destinations do not use this parameter, but retrieve the hostname from the /etc/tnsnames.ora file.
indexes string list "date", "facility", "host", "program" The list of columns that are indexed by the database to speed up searching.
log_fifo_size() number Use global setting. The number of entries in the output buffer (output fifo).
log_disk_fifo_size() number 0 Size of the hard disk space in bytes that is used as disk buffer. Available only in syslog-ng Premium Edition. See Section 8.4, “Using disk-based buffering” for details on using the disk buffer.
password string n/a Password of the database user.
table string n/a Name of the database table to use (can include macros). When using macros, note that some databases limit the length of table names.
type freetds, mysql, oracle, pgsql, or 3 n/a Specifies the type of the database, i.e., the DBI database driver to use. Use the freetds option to send logs to an MSSQL database. See the examples of the databases on the following sections for details.
username string n/a Name of the database user.
values string list "${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}", "$FACILITY", "$LEVEL", "$HOST", "$PROGRAM", "$PID", "$MSGONLY" The values to store in the fields specified in the columns parameter.

Table 9.8. Options for sql()


[Warning] Warning

When using macros in table names, note that some databases limit the maximum allowed length of table names. Consult the documentation of the database for details.

[Example] Example 9.12. Using the sql() driver

The following example sends the log messages into a PostgreSQL database running on the logserver host. The messages are inserted into the logs database, the name of the table includes the exact date and the name of the host sending the messages. The syslog-ng application automatically creates the required tables and columns, if the user account used to connect to the database has the required privileges.

destination d_sql { 
  sql(type(pgsql)
  host("logserver") username("syslog-ng") password("password")
  database("logs")
  table("messages_${HOST}_${R_YEAR}${R_MONTH}${R_DAY}")
  columns("datetime", "host", "program", "pid", "message")
  values("$R_DATE", "$HOST", "$PROGRAM", "$PID", "$MSGONLY")
  indexes("datetime", "host", "program", "pid", "message"));
  };

9.2.5.1. Using the sql() driver with an Oracle database

The Oracle sql destination has some special aspects that are important to note.

  • Apart from the syslog-ng PE packages, it requires the Oracle Instant Client package from Oracle. See the platform-specific installation instructions in Chapter 4, Installing syslog-ng for details.

  • The hostname of the database server is set in the /etc/tnsnames.ora file, not in the host parameter of the sql() destination.

  • As certain database versions limit the maximum length of table names, macros in the table names should be used with care.

  • In the current version of syslog-ng PE, the types of database columns must be explicitly set for the Oracle destination. The column used to store the text part of the syslog messages should be able to store messages as long as the longest message permitted by syslog-ng, therefore it is usually recommended to use the varchar2 or clob column type. (The maximum length of the messages can be set using the log_msg_size() option.) See the following example for details.

[Example] Example 9.13. Using the sql() driver with an Oracle database

The following example sends the log messages into an Oracle database running on the logserver host, which must be set in the /etc/tnsnames.ora file. The messages are inserted into the LOGS database, the name of the table includes the exact date when the messages were sent. The syslog-ng application automatically creates the required tables and columns, if the user account used to connect to the database has the required privileges.

destination d_sql { 
  sql(type(oracle)
  username("syslog-ng") password("password")
  database("LOGS")
  table("msgs_${R_YEAR}${R_MONTH}${R_DAY}")
  columns("datetime varchar(16)", "host varchar(32)", "program varchar(32)", "pid varchar(8)", "message varchar2")                        
  values("$R_DATE", "$HOST", "$PROGRAM", "$PID", "$MSGONLY")
  indexes("datetime", "host", "program", "pid", "message"));
  };

The Oracle Instant Client retrieves the address of the database server from the /etc/tnsnames.ora file. Edit or create this file as needed for your configuration. A sample is provided below.

LOGS =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)
(HOST = logserver)
(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = EXAMPLE.SERVICE)
)
)

9.2.5.2. Using the sql() driver with a Microsoft SQL database

The freetds database driver can access Microsoft SQL (MSSQL) destinations. This driver has some special aspects that are important to note.

  • The date format used by the MSSQL database must be explicitly set in the /etc/locales.conf file of the syslog-ng server. See the following example for details.

  • As certain database versions limit the maximum length of table names, macros in the table names should be used with care.

  • In the current version of syslog-ng PE, the types of database columns must be explicitly set for the MSSQL destination. The column used to store the text part of the syslog messages should be able to store messages as long as the longest message permitted by syslog-ng. The varchar column type can store maximum 4096 bytes-long messages. The maximum length of the messages can be set using the log_msg_size() option. See the following example for details.

  • Remote access for SQL users must be explicitly enabled on the Microsoft Windows host running the Microsoft SQL Server. See Section 4.8, “Configuring Microsoft SQL Server to accept logs from syslog-ng” for details.

[Example] Example 9.14. Using the sql() driver with an MSSQL database

The following example sends the log messages into an MSSQL database running on the logserver host. The messages are inserted into the syslogng database, the name of the table includes the exact date when the messages were sent. The syslog-ng application automatically creates the required tables and columns, if the user account used to connect to the database has the required privileges.

destination d_mssql { 
sql(type(freetds) host("logserver") port("1433") 
  username("syslogng") password("syslogng") database("syslogng") 
  table("msgs_${R_YEAR}${R_MONTH}${R_DAY}")columns("datetime varchar(16)", "host varchar(32)",
  "program varchar(32)", "pid varchar(8)", "message varchar(4096)")
  values("$R_DATE", "$HOST", "$PROGRAM", "$PID", "$MSGONLY")
  indexes("datetime", "host", "program", "pid", "message"));
  };

The date format used by the MSSQL database must be explicitly set in the /etc/locales.conf file of the syslog-ng server. Edit or create this file as needed for your configuration. A sample is provided below.

[default]
date = "%Y-%m-%d %H:%M:%S"

9.2.6. tcp(), tcp6(), udp(), and udp6(),

This driver sends messages to another host on the local intranet or internet using the UDP or TCP protocol. The tcp6() and udp6() drivers use the IPv6 network protocol.

Both drivers have a single required argument specifying the destination host address, where messages should be sent, and several optional parameters. Note that this differs from source drivers, where local bind address is implied, and none of the parameters are required.

The udp() and udp6() drivers automatically send multicast packets if a multicast destination address is specified. The tcp() and tcp6() drivers do not support multicasting.

Declaration:
tcp(host [options]);
udp(host [options]);
tcp6(host [options]);
udp6(host [options]);

Apart from the common destination options described in Table 9.5, “Common options for destination drivers” these destinations have the following options:

Name Type Default Description
ip_tos number 0 Specifies the Type-of-Service value of outgoing packets.
ip_ttl number 0 Specifies the Time-To-Live value of outgoing packets.
localip() string 0.0.0.0 The IP address to bind to before connecting to target.
localport() number 0 The port number to bind to. Messages are sent from this port.
port() or destport() number 514 The port number to connect to.
spoof_source yes or no no Enables source address spoofing. This means that the host running syslog-ng generates UDP packets with the source IP address matching the original sender of the message. It is useful when you want to perform some kind of preprocessing via syslog-ng then forward messages to your central log management solution with the source address of the original sender. This option only works for UDP destinations though the original message can be received by TCP as well. This option is only available if syslog-ng was compiled using the --enable-spoof-source configuration option.
so_broadcast yes or no no This option controls the SO_BROADCAST socket option required to make syslog-ng send messages to a broadcast address. See the socket(7) manual page for details.
so_rcvbuf number 0 Specifies the size of the socket receive buffer in bytes.
so_sndbuf number 0 Specifies the size of the socket send buffer in bytes.
tls tls options n/a This option sets various TLS specific options like key/certificate files and trusted CA locations. TLS can be used only with the tcp/tcp6 transport protocols. See Section 9.7, “TLS options” for more information.

Table 9.9. Options for udp(), udp6(), tcp() and tcp6()


[Example] Example 9.15. Using the tcp() driver
destination d_tcp { tcp("10.1.2.3" port(1999); localport(999)); };

If name resolution is configured, the hostname of the target server can be used as well.

destination d_tcp { tcp("target_host" port(1999); localport(999)); };

The tcp() and tcp6() drivers of the syslog-ng Premium Edition support disk-based buffering, and have the following option:

Name Type Default Description
log_disk_fifo_size() number 0 Size of the hard disk space in bytes that is used as disk buffer. Available only in syslog-ng Premium Edition. See Section 8.4, “Using disk-based buffering” for details on using the disk buffer.

Table 9.10. Special options for tcp() and tcp6() in syslog-ng Premium Edition


9.2.7. unix-stream() & unix-dgram()

This driver sends messages to a unix socket in either SOCK_STREAM or SOCK_DGRAM mode.

Both drivers have a single required argument specifying the name of the socket to connect to.

Declaration: 
  unix-stream(filename [options]);
  unix-dgram(filename [options]);

Apart from the common destination options described in Table 9.5, “Common options for destination drivers” the unix-stream() and unix-dgram() destinations has the following options:

Name Type Default Description
so_broadcast yes or no no This option controls the SO_BROADCAST socket option required to make syslog-ng send messages to a broadcast address. See the socket(7) manual page for details.
so_rcvbuf number 0 Specifies the size of the socket receive buffer in bytes.
so_sndbuf number 0 Specifies the size of the socket send buffer in bytes.

Table 9.11. Options for unix-stream() and unix-dgram()


[Example] Example 9.16. Using the unix-stream() driver
destination d_unix_stream { unix-stream("/var/run/logs"); };

9.2.8. usertty()

This driver writes messages to the terminal of a logged-in user.

The usertty() driver has a single required argument, specifying a username who should receive a copy of matching messages.

Declaration: 
  usertty(username);

The usertty() does not have any further options; nor can it use the common options available for other destinations..

[Note] Note

The usertty() destination does not support templates.

[Example] Example 9.17. Using the usertty() driver
destination d_usertty { usertty("root"); };

© 2007-2008 BalaBit IT Security
Please send your comments or documentation bugs to: documentation@balabit.com