4.5.5. Customize system logging in PSM

PSM uses the syslog-ng Open Source Edition application (version 3.9) for system logging. Starting with PSM 5 LTS, you can customize its configuration to better integrate PSM into your logging infrastructure. If you are not familiar with syslog-ng Open Source Edition, read how syslog-ng OSE works. Customizing the configuration of syslog-ng Open Source Edition allows you to better integrate the log messages of PSM into your environment, for example, to:

  • change the message format or rename message fields,

  • send the messages to multiple logservers or SIEMs, or to

  • select (filter) which messages to send to your logserver.

Limitations

Note that not every feature described in the syslog-ng Open Source Edition documentation is available on PSM. Typically, features that are only rarely used on logging clients are not available, for example, Java-based destinations like HDFS and Elasticsearch. For a detailed list of available modules, execute the syslog-ng --module-registry command.

Customize the syslog-ng configuration

Do not change the syslog configuration of PSM unless you know exactly what you are doing. Incorrect changes can decrease the performance of PSM, disable system logging, or cause message loss. In particular, note the following points:

  1. Create a PSM configuration snippet in a file. Make sure that the filename ends with .conf. Note that syslog-ng OSE uses the configuration objects defined in these files only if they are used in a log path as well, so make sure to include a log path.

    Do not loop messages. That is, make sure that the a destination does not send a message back to the original source of the message (doing so would cause an infinite loop).

  2. Copy it to the /etc/syslog-ng/conf.d/ directory of the core firmware. (If you are using a high-availability PSM cluster, PSM automatically copies the file to the secondary node as well.)

    Files located in this directory do not cause the PSM configuration to be tainted. PSM automatically includes these files in the configuration of syslog-ng Open Source Edition.

    Do not modify the original configuration files (for example, /etc/syslog-ng/syslog-ng.conf or /etc/syslog-ng/conf.d/message-queue-client.conf).

  3. Verify that the resulting syslog-ng OSE configuration file is syntactically valid. The configuration is valid if executing the following command does not show any syntax errors: syslog-ng --syntax-only --no-caps

  4. Your changes will take effect only after you reload the configuration of syslog-ng Open Source Edition using the following command: syslog-ng-ctl reload

    If there are any errors in the configuration, PSM keeps on using the earlier configuration. In this case, correct the configuration, because if PSM reboots while the syslog-ng OSE configuration is invalid, PSM will not be able to log messages.

Available sources

You can use the following sources in your custom configuration. These sources are defined in the stock configuration file of syslog-ng OSE, and are in regular syslog message format (except for s_message_queue_client).

  • s_core_journal: Logs of the PSM host, including log messages about the audited sessions.

  • s_message_queue_client: Logs about the audited sessions in JSON format.

  • s_slave_boot: Logs from the boot firmware of the secondary node in a high-availability PSM cluster.

  • src: Log messages of local PSM services.

  • src-internal: The internal logs of syslog-ng OSE running on PSM.

Certificates and encrypted connections

If you are using a custom destination that requires a certificate (either to authenticate PSM, or to verify the identity of the logserver). In this case, copy the certificates to PSM into the /etc/syslog-ng/conf.d/ directory. In the custom syslog configuration you cannot use the certificates uploaded to PSM using the web interface.

SIEM integration

Customizing the syslog configuration of PSM allows you to send log messages directly to your SIEM (for example, Splunk), in a format that your SIEM can understand.

Balabit can provide you the configuration files needed to send the log messages of PSM to Splunk in the Splunk Common Information Model (CIM) format. If you are interested, contact the Balabit Support Team.

If you need assistance to use another SIEM format, contact professionalservices@balabit.com.

Examples

The following configuration snippet reads the messages from the built-in s_message_queue_client source, parses the JSON message, and sends the messages to a remote destination using the RFC5424 message format (the body of the message remains in JSON).

parser json {
    json-parser(
        prefix(".scb.")
        template("$MSG")
    );
};

destination d_custom_remote {
    syslog(
        "192.168.1.1"
        transport(tcp)
        port(6514)
        template("$(format-json --key .scb.*)\n")
    );
};

log {
    source(s_message_queue_client);
    parser(json);
    destination(d_custom_remote);
};

A sample log message using the above configuration is the following (line-breaks added for clarity):

{"_scb":
  {
    "username":"frobert",
        "timestamp":"1499673286.380022",
        "session_id":"svc/hm4soFrNYeAmkyzLVkzi1o/ssh_demo:4",
        "server_port":"22",
        "server_ip":"10.30.255.70",
        "protocol":"ssh",
        "event_type":"server_authentication_success",
        "connection_name":"ssh_demo",
        "client_port":"50830",
        "client_ip":"10.30.255.70"
    }
}

To use this configuration snippet on your PSM, copy it to a file (make sure that the filename ends with .conf), change the IP address and port number to match your environment, copy it to the core firmware of your PSM into the /etc/syslog-ng/conf.d directory, then reload the syslog-ng configuration using syslog-ng-ctl reload.