The syslog-ng 3.0 Administrator Guide

The logstore() driver stores log messages in binary files that can be encrypted, compressed, checked for integrity, and timestamped by an external Timestamping Authority (TSA). Otherwise, it is very similar to the file() destination.

Logstore files consist of individual chunks, every chunk can be encrypted, compressed, and timestamped separately. Chunks contain log message data, chunk size defaults to 128k (about 1MB worth of compressed logs).

To display the contents of a logstore file, use the logcat command supplied with syslog-ng, e.g., logcat /var/log/messages.lgs. To display the contents of encrypted log files, specify the private key of the certificate used to encrypt the file, e.g., logcat -k private.key /var/log/messages.lgs. The contents of the file are sent to the standard output, so it is possible to use grep and other tools to find particular log messages, e.g., logcat /var/log/messages.lgs |grep 192.168.1.1.

Every record that is stored in the logstore has a unique record ID. The logcat application can quickly jump to a specified record using the -- seek option.

For files that are in use by syslog-ng, the last chunk that is open cannot be read. Chunks are closed when their size reaches the limit set in the chunk_size parameter, or when the time limit set in the chunk_time parameter expires and no new message arrives.

The syslog-ng PE application generates an SHA-1 hash for every chunk to verify the integrity of the chunk. The hashes of the chunks are chained together to prevent injecting chunks into the logstore file. The syslog-ng application can encrypt the logstore using the aes128 algorithm in CBC mode; the hashing (HMAC) algorithm is hmac-sha1. Currently it is not possible to use other algorithms.

	Warning
	If the syslog-ng Premium Edition application or the computer crashes, an unclosed chunk remains at the end of the file. This chunk is marked as broken, its data stays there but is not shown by logcat.

The destination filename may include macros which get expanded when the message is written, thus a simple logstore() driver may create several files. For more information on available macros see Section 8.5, “Macros”.

If the expanded filename refers to a directory which does not exist, it will be created depending on the create_dirs() setting (both global and a per destination option).

The logstore() has a single required parameter that specifies the filename that stores the log messages. For the list of available optional parameters, see Section 8.2.2, “logstore()”.

Declaration:
                logstore(filename options());

Example 3.19. Using the logstore() driver

A simple example saving and compressing log messages. destination d_logstore { logstore("/var/log/messages.lgs" compress(5) ); }; A more detailed example that encrypts messages, modifies the parameters for closing chunks, and sets file privileges. destination d_logstore { logstore("/var/log/messages-logstore.lgs" encrypt_certificate("/opt/syslog-ng/etc/syslog-ng/keys/10-100-20-40/public-certificate-of-the-server.pem") chunk_size(100) chunk_time(5) owner("balabit") group("balabit") perm(0777) ); };

	Note
	When using the logstore() destination, update the configuration of your log rotation program to rotate these files. Otherwise, the log files can become very large.

Warning

Since the state of each created file must be tracked by syslog-ng, it consumes some memory for each file. If no new messages are written to a file within 60 seconds (controlled by the time_reap() global option), it is closed, and its state is freed. Exploiting this, a DoS attack can be mounted against the system. If the number of possible destination files and its needed memory is more than the amount available on the syslog-ng server. The most suspicious macro is $PROGRAM, where the number of possible variations is rather high. Do not use the $PROGRAM macro in insecure environments.