The syslog-ng Store Box 4 F7 Administrator Guide

Copyright © 2017 Balabit SA. All rights reserved. This document is protected by copyright and is distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this document may be reproduced in any form by any means without prior written authorization of Balabit.

This documentation and the product it describes are considered protected by copyright according to the applicable laws.

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (https://www.openssl.org/). This product includes cryptographic software written by Eric Young (eay@cryptsoft.com)

AIX™, AIX 5L™, AS/400™, BladeCenter™, eServer™, IBM™, the IBM™ logo, IBM System i™, IBM System i5™, IBM System x™, iSeries™, i5/OS™, Netfinity™, NetServer™, OpenPower™, OS/400™, PartnerWorld™, POWER™, ServerGuide™, ServerProven™, and xSeries™ are trademarks or registered trademarks of International Business Machines.

Alliance Log Agent for System i™ is a registered trademark of Patrick Townsend & Associates, Inc.

Amazon Web Services™ and the “Powered by Amazon Web Services” logo are trademarks of Amazon.com, Inc. or its affiliates in the United States and/or other countries.

The Balabit™ name and the Balabit™ logo are registered trademarks of Balabit SA.

Debian™ is a registered trademark of Software in the Public Interest Inc.

Linux™ is a registered trademark of Linus Torvalds.

MySQL™ is a registered trademark of Oracle and/or its affiliates.

Oracle™, JD Edwards™, PeopleSoft™, and Siebel™ are registered trademarks of Oracle Corporation and/or its affiliates.

Red Hat™, Inc., Red HatEnterprise Linux™ and Red HatLinux™ are trademarks of Red Hat, Inc.

SUSE™ is a trademark of SUSE AG, a Novell business.

Solaris™ is a registered trademark of Oracle and/or its affiliates.

Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries.

The syslog-ng™ name and the syslog-ng™ logo are registered trademarks of Balabit.

VMware™, VMware ESX™ and VMware View™ are trademarks or registered trademarks of VMware, Inc. and/or its affiliates.

Windows™ 95, 98, ME, 2000, XP, Server 2003, Vista, Server 2008, 7, 8, and Server 2012 are registered trademarks of Microsoft Corporation.

All other product names mentioned herein are the trademarks of their respective owners.

DISCLAIMER. Balabit is not responsible for any third-party websites mentioned in this document. Balabit does not endorse and is not responsible or liable for any content, advertising, products, or other material on or available from such sites or resources. Balabit will not be responsible or liable for any damage or loss caused or alleged to be caused by or in connection with use of or reliance on any such content, goods, or services that are available on or through any such sites or resources.

July 24, 2017

This document is the primary manual of the syslog-ng Store Box 4 F7.


Table of Contents

Preface
1. Summary of contents
2. Target audience and prerequisites
3. Products covered in this guide
4. Typographical conventions
5. Contact and support information
5.1. Sales contact
5.2. Support contact
5.3. Training
6. About this document
6.1. Summary of changes
6.2. Feedback
1. Introduction
1.1. What SSB is
1.2. What SSB is not
1.3. Why is SSB needed
1.4. Who uses SSB
2. The concepts of SSB
2.1. The philosophy of SSB
2.2. Collecting logs with SSB
2.3. Managing incoming and outgoing messages with flow-control
2.4. Receiving logs from a secure channel
2.5. Reliable Log Transfer Protocol
2.6. Network interfaces
2.7. High Availability support in SSB
2.8. Firmware in SSB
2.8.1. Firmware and high availability
2.9. Versions and releases of SSB
2.10. Licensing model and modes of operation
2.10.1. Notes about counting the licensed hosts
2.11. Licensing benefits
2.12. License types
2.12.1. Perpetual license
2.12.2. Subscription-based license
2.13. Licensing examples
2.14. The structure of a log message
2.14.1. BSD-syslog or legacy-syslog messages
2.14.2. IETF-syslog messages
3. The Welcome Wizard and the first login
3.1. The initial connection to SSB
3.1.1. Creating an alias IP address (Microsoft Windows)
3.1.2. Creating an alias IP address (Linux)
3.1.3. Modifying the IP address of SSB
3.2. Configuring SSB with the Welcome Wizard
4. Basic settings
4.1. Supported web browsers and operating systems
4.2. The structure of the web interface
4.2.1. Elements of the main workspace
4.2.2. Multiple web users and locking
4.2.3. Web interface and RPC API
4.3. Network settings
4.3.1. Configuring the management interface
4.3.2. Configuring the routing table
4.4. Date and time configuration
4.4.1. Configuring a time (NTP) server
4.5. SNMP and e-mail alerts
4.5.1. Configuring e-mail alerts
4.5.2. Configuring SNMP alerts
4.5.3. Querying SSB status information using agents
4.6. Configuring system monitoring on SSB
4.6.1. Configuring monitoring
4.6.2. Health monitoring
4.6.3. Preventing disk space fill up
4.6.4. Configuring message rate alerting
4.6.5. System related traps
4.6.6. Alerts related to syslog-ng
4.7. Data and configuration backups
4.7.1. Creating a backup policy using Rsync over SSH
4.7.2. Creating a backup policy using SMB/CIFS
4.7.3. Creating a backup policy using NFS
4.7.4. Creating configuration backups
4.7.5. Creating data backups
4.7.6. Encrypting configuration backups with GPG
4.8. Archiving and cleanup
4.8.1. Creating a cleanup policy
4.8.2. Creating an archive policy using SMB/CIFS
4.8.3. Creating an archive policy using NFS
4.8.4. Archiving or cleaning up the collected data
5. User management and access control
5.1. Managing SSB users locally
5.1.1. Creating local users in SSB
5.1.2. Deleting a local user from SSB
5.2. Setting password policies for local users
5.3. Managing local usergroups
5.4. Managing SSB users from an LDAP database
5.5. Authenticating users to a RADIUS server
5.6. Managing user rights and usergroups
5.6.1. Assigning privileges to usergroups for the SSB web interface
5.6.2. Modifying group privileges
5.6.3. Finding specific usergroups
5.6.4. How to use usergroups
5.6.5. Built-in usergroups of SSB
5.7. Listing and searching configuration changes
6. Managing SSB
6.1. Controlling SSB — restart, shutdown
6.2. Managing a high availability SSB cluster
6.2.1. Adjusting the synchronization speed
6.2.2. Asynchronous data replication
6.2.3. Redundant heartbeat interfaces
6.2.4. Next-hop router monitoring
6.3. Upgrading SSB
6.3.1. Upgrade checklist
6.3.2. Upgrading SSB (single node)
6.3.3. Upgrading an SSB cluster
6.3.4. Troubleshooting
6.3.5. Reverting to an older firmware version
6.3.6. Updating the SSB license
6.3.7. Exporting the configuration of SSB
6.3.8. Importing the configuration of SSB
6.4. Accessing the SSB console
6.4.1. Using the console menu of SSB
6.4.2. Enabling SSH access to the SSB host
6.4.3. Changing the root password of SSB
6.5. Sealed mode
6.5.1. Disabling sealed mode
6.6. Out-of-band management of SSB
6.6.1. Configuring the IPMI interface
6.7. Managing the certificates used on SSB
6.7.1. Generating certificates for SSB
6.7.2. Uploading external certificates to SSB
6.7.3. Generating TSA certificate with Windows Certificate Authority
6.8. Creating hostlist policies
6.8.1. Creating hostlists
6.8.2. Importing hostlists from files
7. Configuring message sources
7.1. Default message sources in SSB
7.2. Receiving SNMP messages
7.3. Creating syslog message sources in SSB
7.4. Creating SQL message sources in SSB
7.4.1. Fetching the SQL database
7.4.2. Configuring message parts in Basic mode
7.4.3. Configuring message parts in Advanced mode
7.4.4. Creating a fetch query manually
8. Storing messages on SSB
8.1. Using logstores
8.1.1. Creating logstores
8.1.2. Configuring the indexer service
8.1.3. Viewing encrypted logs with logcat
8.2. Creating text logspaces
8.3. Managing logspaces
8.4. Creating filtered logspaces
8.5. Creating remote logspaces
8.6. Creating multiple logspaces
8.7. Accessing log files across the network
8.7.1. Sharing log files in standalone mode
8.7.2. Sharing log files in domain mode
8.7.3. Accessing shared files
9. Forwarding messages from SSB
9.1. Forwarding log messages to SQL databases
9.2. SQL templates in SSB
9.2.1. The Legacy template
9.2.2. The Full template
9.2.3. The Custom template
9.3. Forwarding log messages to remote servers
9.4. Forwarding log messages to SNMP destinations
9.5. Using SSB as a relay
10. Log paths — routing and processing messages
10.1. Default logpaths in SSB
10.2. Creating new log paths
10.3. Filtering messages
10.4. Modifying messages using rewrite
10.5. Parsing sudo log messages
10.6. Parsing key-value pairs
11. Configuring syslog-ng options
11.1. General syslog-ng settings
11.2. Timestamping configuration on SSB
11.3. Using name resolution on SSB
11.4. Setting the certificates used in TLS-encrypted log transport
12. Searching log messages
12.1. Using the search interface
12.1.1. Customizing columns of the log message search interface
12.1.2. Metadata collected about log messages
12.1.3. Using complex search queries
12.2. Browsing encrypted logspaces
12.2.1. Using persistent decryption keys
12.2.2. Using session-only decryption keys
12.2.3. Assigning decryption keys to a logstore
12.3. Creating custom statistics from log data
12.3.1. Displaying log statistics
12.3.2. Creating reports from custom statistics
12.4. Creating content-based alerts
12.4.1. Setting up alerts on the search interface
12.4.2. Setting up alerts on the Search > Content-Based Alerts page
12.4.3. Format of alert messages
12.5. Additional tools
13. Searching the internal messages of SSB
13.1. Using the internal search interfaces
13.1.1. Filtering
13.1.2. Exporting the results
13.1.3. Customizing columns of the internal search interfaces
13.2. Changelogs of SSB
13.3. Configuration changes of syslog-ng peers
13.4. Log message alerts
13.5. Notifications on archiving and backups
13.6. Status history and statistics
13.6.1. Displaying custom syslog-ng statistics
13.6.2. Statistics collection options
13.7. Reports
13.7.1. Contents of the default reports
13.7.2. Generating partial reports
13.7.3. Configuring custom reports
14. Classifying messages with pattern databases
14.1. The structure of the pattern database
14.2. How pattern matching works
14.3. Searching for rulesets
14.4. Creating new rulesets and rules
14.5. Exporting databases and rulesets
14.6. Importing pattern databases
14.7. Using pattern parsers
14.8. Using parser results in filters and templates
14.9. Using the values of pattern parsers in filters and templates
15. The SSB RPC API
15.1. Requirements for using the RPC API
15.2. RPC client requirements
15.3. Documentation of the RPC API
16. Troubleshooting SSB
16.1. Network troubleshooting
16.2. Gathering data about system problems
16.3. Viewing logs on SSB
16.4. Collecting logs and system information for error reporting
16.5. Troubleshooting an SSB cluster
16.5.1. Understanding SSB cluster statuses
16.5.2. Recovering SSB if both nodes broke down
16.5.3. Recovering from a split brain situation
16.5.4. Replacing a node in an SSB HA cluster
16.5.5. Resolving an IP conflict between cluster nodes
16.6. Restoring SSB configuration and data
A. Security checklist for configuring SSB
B. END USER LICENSE AGREEMENT FOR BALABIT PRODUCT (EULA)
Glossary
Index
List of SSB web interface labels

List of Procedures

2.2. Collecting logs with SSB
3.1.1. Creating an alias IP address (Microsoft Windows)
3.1.2. Creating an alias IP address (Linux)
3.1.3. Modifying the IP address of SSB
3.2. Configuring SSB with the Welcome Wizard
4.3.1. Configuring the management interface
4.3.2. Configuring the routing table
4.4.1. Configuring a time (NTP) server
4.5.1. Configuring e-mail alerts
4.5.2. Configuring SNMP alerts
4.5.3. Querying SSB status information using agents
4.6.1. Configuring monitoring
4.6.3. Preventing disk space fill up
4.6.4. Configuring message rate alerting
4.7.1. Creating a backup policy using Rsync over SSH
4.7.2. Creating a backup policy using SMB/CIFS
4.7.3. Creating a backup policy using NFS
4.7.4. Creating configuration backups
4.7.5. Creating data backups
4.7.6. Encrypting configuration backups with GPG
4.8.1. Creating a cleanup policy
4.8.2. Creating an archive policy using SMB/CIFS
4.8.3. Creating an archive policy using NFS
4.8.4. Archiving or cleaning up the collected data
5.1.1. Creating local users in SSB
5.1.2. Deleting a local user from SSB
5.2. Setting password policies for local users
5.3. Managing local usergroups
5.4. Managing SSB users from an LDAP database
5.5. Authenticating users to a RADIUS server
5.6.1. Assigning privileges to usergroups for the SSB web interface
5.6.2. Modifying group privileges
6.2.3. Redundant heartbeat interfaces
6.2.4. Next-hop router monitoring
6.3.2. Upgrading SSB (single node)
6.3.3. Upgrading an SSB cluster
6.3.5. Reverting to an older firmware version
6.3.6. Updating the SSB license
6.3.7. Exporting the configuration of SSB
6.3.8. Importing the configuration of SSB
6.4.2. Enabling SSH access to the SSB host
6.4.3. Changing the root password of SSB
6.5.1. Disabling sealed mode
6.6.1. Configuring the IPMI interface
6.7.1. Generating certificates for SSB
6.7.2. Uploading external certificates to SSB
6.7.3. Generating TSA certificate with Windows Certificate Authority
6.8.1. Creating hostlists
6.8.2. Importing hostlists from files
7.2. Receiving SNMP messages
7.3. Creating syslog message sources in SSB
7.4.1. Fetching the SQL database
7.4.2. Configuring message parts in Basic mode
7.4.3. Configuring message parts in Advanced mode
8.1.1. Creating logstores
8.1.2. Configuring the indexer service
8.2. Creating text logspaces
8.4. Creating filtered logspaces
8.5. Creating remote logspaces
8.6. Creating multiple logspaces
8.7.1. Sharing log files in standalone mode
8.7.2. Sharing log files in domain mode
9.1. Forwarding log messages to SQL databases
9.3. Forwarding log messages to remote servers
9.4. Forwarding log messages to SNMP destinations
9.5. Using SSB as a relay
10.2. Creating new log paths
10.4. Modifying messages using rewrite
10.5. Parsing sudo log messages
10.6. Parsing key-value pairs
11.4. Setting the certificates used in TLS-encrypted log transport
12.1.1. Customizing columns of the log message search interface
12.2.1. Using persistent decryption keys
12.2.2. Using session-only decryption keys
12.2.3. Assigning decryption keys to a logstore
12.3.2. Creating reports from custom statistics
12.4.1. Setting up alerts on the search interface
12.4.2. Setting up alerts on the Search > Content-Based Alerts page
13.1.3. Customizing columns of the internal search interfaces
13.6.1. Displaying custom syslog-ng statistics
13.7.2. Generating partial reports
13.7.3. Configuring custom reports
14.4. Creating new rulesets and rules
14.8. Using parser results in filters and templates
16.1. Network troubleshooting
16.3. Viewing logs on SSB
16.4. Collecting logs and system information for error reporting
16.5.2. Recovering SSB if both nodes broke down
16.5.3. Recovering from a split brain situation
16.5.4. Replacing a node in an SSB HA cluster
16.5.5. Resolving an IP conflict between cluster nodes
16.6. Restoring SSB configuration and data