19.2. Handling large message load

This section provides tips on optimizing the performance of syslog-ng. Optimizing the performance is important for syslog-ng hosts that handle large traffic.

  • Disable DNS resolution, or resolve hostnames locally. For details, see Section 19.3, Using name resolution in syslog-ng.

  • Enable flow-control for the TCP sources. For details, see Section 8.2, Managing incoming and outgoing messages with flow-control.

  • Do not use the usertty() destination driver. Under heavy load, the users are not be able to read the messages from the console, and it slows down syslog-ng.

  • Do not use regular expressions in our filters. Evaluating general regular expressions puts a high load on the CPU. Use simple filter functions and logical operators instead. For details, see Section 11.3, Regular expressions.

  • Warning

    When receiving messages using the UDP protocol, increase the size of the UDP receive buffer on the receiver host (that is, the syslog-ng OSE server or relay receiving the messages). Note that on certain platforms, for example, on Red Hat Enterprise Linux 5, even low message load (~200 messages per second) can result in message loss, unless the so-rcvbuf() option of the source is increased. In such cases, you will need to increase the net.core.rmem_max parameter of the host (for example, to 1024000), but do not modify net.core.rmem_default parameter.

    As a general rule, increase the so-rcvbuf() so that the buffer size in kilobytes is higher than the rate of incoming messages per second. For example, to receive 2000 messages per second, set the so-rcvbuf() at least to 2 097 152 bytes.

  • Increase the value of the flush-lines() parameter. Increasing flush-lines() from 0 to 100 can increase the performance of syslog-ng OSE by 100%.