Table of Contents

Preface

1. Summary of contents

2. Target audience and prerequisites

3. Products covered in this guide

4. Typographical conventions

5. Contact and support information

5.1. Sales contact

5.2. Support contact

5.3. Training

6. About this document

6.1. Summary of changes

6.2. Feedback

6.3. Acknowledgments

1. Introduction to syslog-ng

1.1. What syslog-ng is

1.2. What syslog-ng is not

1.3. Why is syslog-ng needed?

1.4. What is new in syslog-ng Premium Edition 4 LTS?

1.5. Who uses syslog-ng?

1.6. Supported platforms

1.6.1. Certified packages

2. The concepts of syslog-ng

2.1. The philosophy of syslog-ng

2.2. Logging with syslog-ng

2.2.1. The route of a log message in syslog-ng

2.3. Modes of operation

2.3.1. Client mode

2.3.2. Relay mode

2.3.3. Server mode

2.4. Global objects

2.5. Timezones and daylight saving

2.5.1. A note on timezones and timestamps

2.6. Versions and releases of syslog-ng PE

2.7. Licensing

2.8. High availability support

2.9. The structure of a log message

2.9.1. BSD-syslog or legacy-syslog messages

2.9.2. IETF-syslog messages

2.9.3. Message representation in syslog-ng PE

3. Installing syslog-ng

3.1. Installing syslog-ng using the .run installer

3.1.1. Installing syslog-ng in client or relay mode

3.1.2. Installing syslog-ng in server mode

3.1.3. Installing syslog-ng without user-interaction

3.2. Installing syslog-ng on RPM-based platforms (Red Hat, SUSE, AIX)

3.3. Installing syslog-ng on Debian-based platforms

3.4. Enabling SELinux support

3.5. Upgrading syslog-ng PE

3.5.1. Upgrading syslog-ng PE to other package versions

3.5.2. Uprading from previous syslog-ng PE versions

3.5.3. Upgrading from previous syslog-ng OSE versions

3.5.4. Upgrading from syslog-ng PE to syslog-ng OSE

3.5.5. Upgrading from complete syslog-ng PE to client setup version of syslog-ng PE

3.6. Uninstalling syslog-ng PE

3.7. Configuring Microsoft SQL Server to accept logs from syslog-ng

4. The syslog-ng PE quick-start guide

4.1. Configuring syslog-ng on client hosts

4.2. Configuring syslog-ng on server hosts

4.3. Configuring syslog-ng relays

4.3.1. Configuring syslog-ng on relay hosts

4.3.2. How relaying log messages works

5. The syslog-ng PE configuration file

5.1. The syslog-ng configuration file

5.1.1. The configuration syntax in detail

5.1.2. Notes about the configuration syntax

5.2. Logging configuration changes

5.3. Managing large syslog-ng configurations

5.3.1. Including configuration files

6. Collecting log messages — sources and source drivers

6.1. How sources work

6.2. Collecting internal messages

6.3. Collecting messages from text files

6.3.1. File sources and the RFC5424 message format

6.3.2. file() source options

6.4. Collecting messages from named pipes

6.4.1. pipe() source options

6.5. Receiving messages from external applications

6.5.1. program() source options

6.6. Collecting messages on Sun Solaris

6.6.1. sun-streams() source options

6.7. Collecting messages using the IETF syslog protocol

6.7.1. syslog() source options

6.8. Collecting the system-specific log messages of a platform

6.9. Collecting messages from remote hosts using the BSD syslog protocol

6.9.1. tcp(), tcp6(), udp() and udp6() source options

6.10. Collecting messages from UNIX domain sockets

6.10.1. unix-stream() and unix-dgram() source options

7. Sending and storing log messages — destinations and destination drivers

7.1. Storing messages in plain-text files

7.1.1. file() destination options

7.2. Storing messages in encrypted files

7.2.1. Displaying the contents of logstore files

7.2.2. Journal files

7.2.3. logstore() destination options

7.3. Sending messages to named pipes

7.3.1. pipe() destination options

7.4. Sending messages to external applications

7.4.1. program() destination options

7.5. Storing messages in an SQL database

7.5.1. Using the sql() driver with an Oracle database

7.5.2. Using the sql() driver with a Microsoft SQL database

7.5.3. The way syslog-ng interacts with the database

7.5.4. sql() destination options

7.6. Sending messages to a remote logserver using the IETF-syslog protocol

7.6.1. syslog() destination options

7.7. Sending messages to a remote logserver using the legacy BSD-syslog protocol

7.7.1. tcp(), tcp6(), udp(), and udp6() destination options

7.8. Sending messages to UNIX domain sockets

7.8.1. unix-stream() and unix-dgram() destination options

7.9. Sending messages to a user terminal — usertty() destination

8. Routing messages: log paths, reliability, and filters

8.1. Log paths

8.1.1. Embedded log statements

8.1.2. Log path flags

8.2. Managing incoming and outgoing messages with flow-control

8.2.1. Flow-control and multiple destinations

8.2.2. Configuring flow-control

8.3. Using disk-based buffering

8.3.1. Enabling disk-based buffering

8.4. Client-side failover

8.5. Filters

8.5.1. Using filters

8.5.2. Combining filters with boolean operators

8.5.3. Using wildcards, special characters, and regular expressions in filters

8.5.4. Tagging messages

8.5.5. Filter functions

8.6. Dropping messages

9. Global options of syslog-ng PE

9.1. Configuring global syslog-ng options

9.2. Global options

10. TLS-encrypted message tranfer

10.1. Secure logging using TLS

10.2. Encrypting log messages with TLS

10.2.1. Configuring TLS on the syslog-ng clients

10.2.2. Configuring TLS on the syslog-ng server

10.3. Mutual authentication using TLS

10.3.1. Configuring TLS on the syslog-ng clients

10.3.2. Configuring TLS on the syslog-ng server

10.4. TLS options

11. Manipulating messages

11.1. Customizing message format

11.1.1. Formatting messages, filenames, directories, and tablenames

11.1.2. Templates and macros

11.1.3. Hard vs. soft macros

11.1.4. Macros of syslog-ng PE

11.2. Modifying messages

11.3. Regular expressions

11.3.1. Types and options of regular expressions

11.3.2. Optimizing regular expressions

12. Parsing and segmenting structured messages

12.1. Parsing messages

12.2. Options of CSV parsers

13. Processing message content with a pattern database

13.1. Classifying log messages

13.1.1. The structure of the pattern database

13.1.2. How pattern matching works

13.1.3. Artificial ignorance

13.2. Using pattern databases

13.2.1. Using parser results in filters and templates

13.2.2. Downloading sample pattern databases

13.3. Creating pattern databases

13.3.1. Using pattern parsers

13.3.2. The syslog-ng pattern database format

14. Statistics of syslog-ng

15. Troubleshooting syslog-ng

15.1. Possible causes of losing log messages

15.2. Creating syslog-ng core files

15.3. Collecting debugging information with strace, truss, or tusc

15.4. Running a failure script

15.5. Stopping syslog-ng

16. Best practices and examples

16.1. General recommendations

16.2. Handling lots of parallel connections

16.3. Handling large message load

16.4. Using name resolution in syslog-ng

16.4.1. Resolving hostnames locally

16.5. Collecting logs from chroot

Appendix 1. The syslog-ng manual pages

dqtool — Display the contents of a disk-buffer file created with syslog-ng Premium Edition

loggen — Generate syslog messages at a specified rate

lgstool — Inspect and validate the binary log files (logstores) created with syslog-ng Premium Edition

pdbtool — An application to test and convert syslog-ng pattern database rules

syslog-ng — syslog-ng system logger application

syslog-ng.conf — syslog-ng configuration file

syslog-ng-ctl — Display message statistics and enable verbose, debug and trace modes in syslog-ng Premium Edition

Appendix 2. BalaBit syslog-ng Premium Edition License contract

2.1. SUBJECT OF THE LICENSE CONTRACT

2.2. DEFINITIONS

2.3. WORDS AND EXPRESSIONS

2.4. LICENSE GRANTS AND RESTRICTIONS

2.5. SUBSIDIARIES

2.6. INTELLECTUAL PROPERTY RIGHTS

2.7. TRADE MARKS

2.8. NEGLIGENT INFRINGEMENT

2.9. INTELLECTUAL PROPERTY INDEMNIFICATION

2.10. LICENSE FEE

2.11. WARRANTIES

2.12. DISCLAIMER OF WARRANTIES

2.13. LIMITATION OF LIABILITY

2.14. DURATION AND TERMINATION

2.15. AMENDMENTS

2.16. WAIVER

2.17. SEVERABILITY

2.18. NOTICES

2.19. MISCELLANEOUS

Appendix 3. GNU Lesser General Public License

3.1. Preamble

3.2. TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

3.2.1. Section 0

3.2.2. Section 1

3.2.3. Section 2

3.2.4. Section 3

3.2.5. Section 4

3.2.6. Section 5

3.2.7. Section 6

3.2.8. Section 7

3.2.9. Section 8

3.2.10. Section 9

3.2.11. Section 10

3.2.12. Section 11

3.2.13. Section 12

3.2.14. Section 13

3.2.15. Section 14

3.2.16. NO WARRANTY Section 15

3.2.17. Section 16

3.3. How to Apply These Terms to Your New Libraries

Appendix 4. GNU General Public License

4.1. Preamble

4.2. TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

4.2.1. Section 0

4.2.2. Section 1

4.2.3. Section 2

4.2.4. Section 3

4.2.5. Section 4

4.2.6. Section 5

4.2.7. Section 6

4.2.8. Section 7

4.2.9. Section 8

4.2.10. Section 9

4.2.11. Section 10

4.2.12. NO WARRANTY Section 11

4.2.13. Section 12

4.3. How to Apply These Terms to Your New Programs

Appendix 5. Creative Commons Attribution Non-commercial No Derivatives (by-nc-nd) License

Glossary

List of syslog-ng PE parameters

Index