The syslog-ng Administrator Guide

Between the application and the syslog-ng client: Make sure to use an appropriate source to receive the logs from the application (e.g., from /dev/log). For example, use unix-stream instead of unix-dgram whenever possible.

When syslog-ng is sending messages: If syslog-ng cannot send messages to the destination and the output buffer gets full, syslog-ng will drop messages. The number of dropped messages is displayed per destination in the log message statistics of syslog-ng (see Section 9.1.3.1, “Log statistics” for details). To prevent such message loss, use the disk buffer of syslog-ng Premium Edition to increase the capacity of your output buffer beyond that would be feasible using only a memory-based buffer.

On the network: When transferring messages using the UDP protocol, messages may be lost without any notice or feedback — such is the nature of the UDP protocol. Always use the TCP protocol to transfer messages over the network whenever possible.

In the socket receive buffer: When transferring messages using the UDP protocol, the UDP datagram (i.e., the message) that reaches the receiving host placed in a memory area called the socket receive buffer. If the host receives more messages than it can process, this area overflows, and the kernel drops messages without letting syslog-ng know about it. Using TCP instead of UDP prevents this issue. If you must use the UDP protocol, increase the size of the receive buffer using the so_rcvbuf() option.

When syslog-ng is receiving messages: The receiving syslog-ng (e.g., the syslog-ng server or relay) may drop messages if the fifo of the destination file gets full. The number of dropped messages is displayed per destination in the log message statistics of syslog-ng (see Section 9.1.3.1, “Log statistics” for details). To prevent such message loss, adjust the fifo appropriately for the message load and use the disk buffer of syslog-ng Premium Edition. See Section 8.2, “Handling large message load” and Section 8.4, “Using disk-based buffering” for details.

When the destination cannot handle large load: When syslog-ng is sending messages at a high rate into an SQL database, a file, or another destination, it is possible that the destination cannot handle the load, and processes the messages slowly. As a result, the buffers of syslog-ng fill up, syslog-ng cannot process the incoming messages, and starts to loose messages. See the previous entry for details. Use the throttle parameter (see Section 9.2.1, “Options common for every destination”) and the disk buffer of syslog-ng Premium Edition (Section 8.4, “Using disk-based buffering”).