Table of Contents

Preface

1. Summary of contents

2. Target audience and prerequisites

3. Products covered in this guide

4. Typographical conventions

5. Contact and support information

5.1. Sales contact

5.2. Support contact

5.3. Training

6. About this document

6.1. What is new in this main edition of The syslog-ng Premium Edition Administrator Guide?

6.2. Summary of changes

6.3. Feedback

6.4. Acknowledgments

1. Introduction to syslog-ng

1.1. What syslog-ng is

1.2. What syslog-ng is not

1.3. Why is syslog-ng needed?

1.4. What is new in syslog-ng Premium Edition 3.2?

1.5. Who uses syslog-ng?

1.6. Supported platforms

2. The concepts of syslog-ng

2.1. The philosophy of syslog-ng

2.2. Logging with syslog-ng

2.2.1. The route of a log message in syslog-ng

2.2.2. Embedded log statements

2.3. Modes of operation

2.3.1. Client mode

2.3.2. Relay mode

2.3.3. Server mode

2.4. Global objects

2.5. Timezone handling

2.6. Daylight saving changes

2.7. Secure logging using TLS

2.8. Secure storage of log messages

2.8.1. Journal files

2.9. Formatting messages, filenames, directories, and tablenames

2.10. Segmenting messages

2.11. Modifying messages

2.12. Classifying log messages

2.12.1. The structure of the pattern database

2.12.2. Pattern matching

2.12.3. Artificial ignorance

2.13. Managing incoming and outgoing messages with flow-control

2.13.1. Flow-control and multiple destinations

2.14. Using disk-based buffering

2.15. Client-side failover

2.16. Stable and feature releases of syslog-ng PE

2.17. Licensing

2.18. High availability support

2.19. Possible causes of losing log messages

2.20. The structure of a log message

2.20.1. BSD-syslog or legacy-syslog messages

2.20.2. IETF-syslog messages

3. Installing syslog-ng

3.1. Installing syslog-ng using the .run installer

3.1.1. Installing syslog-ng in client or relay mode

3.1.2. Installing syslog-ng in server mode

3.1.3. Installing syslog-ng without user-interaction

3.2. Installing syslog-ng on RPM-based platforms (Red Hat, SUSE, AIX)

3.3. Installing syslog-ng on Debian-based platforms

3.4. Uninstalling syslog-ng

3.5. Configuring Microsoft SQL Server to accept logs from syslog-ng

4. Configuring syslog-ng

4.1. The syslog-ng configuration file

4.1.1. Including configuration files

4.1.2. Logging configuration changes

4.2. Defining global objects

4.2.1. Notes about the configuration syntax

4.3. Sources and source drivers

4.3.1. Collecting internal messages

4.3.2. Collecting messages from text files

4.3.3. Collecting messages from named pipes

4.3.4. Collecting messages on Sun Solaris

4.3.5. Collecting messages using the IETF syslog protocol

4.3.6. Collecting messages from remote hosts using the BSD syslog protocol

4.3.7. Collecting messages from UNIX domain sockets

4.4. Destinations and destination drivers

4.4.1. Storing messages in plain-text files

4.4.2. Storing messages in encrypted files

4.4.3. Sending messages to named pipes

4.4.4. Sending messages to external applications

4.4.5. Storing messages in an SQL database

4.4.6. Sending messages to a remote logserver using the IETF-syslog protocol

4.4.7. Sending messages to a remote logserver using the legacy BSD-syslog protocol

4.4.8. Sending messages to UNIX domain sockets

4.4.9. usertty()

4.5. Log paths

4.5.1. Using embedded log statements

4.5.2. Configuring flow-control

4.6. Filters

4.6.1. Using filters

4.6.2. Optimizing regular expressions in filters

4.6.3. Tagging messages

4.7. Templates and macros

4.8. Parsing messages

4.9. Classifying messages

4.9.1. Downloading sample pattern databases

4.9.2. Using parser results in filters and templates

4.10. Rewriting messages

4.11. Configuring global syslog-ng options

4.12. Enabling disk-based buffering

4.13. Encrypting log messages with TLS

4.13.1. Configuring TLS on the syslog-ng clients

4.13.2. Configuring TLS on the syslog-ng server

4.14. Mutual authentication using TLS

4.14.1. Configuring TLS on the syslog-ng clients

4.14.2. Configuring TLS on the syslog-ng server

4.15. Configuring syslog-ng on client hosts

4.16. Configuring syslog-ng on relay hosts

4.17. Configuring syslog-ng on server hosts

4.18. Installing and upgrading the license

4.19. Troubleshooting syslog-ng

4.19.1. Creating syslog-ng core files

4.19.2. Running a failure script

4.19.3. Stopping syslog-ng

5. Best practices and examples

5.1. General recommendations

5.2. Handling lots of parallel connections

5.3. Handling large message load

5.4. Using name resolution in syslog-ng

5.4.1. Resolving hostnames locally

5.5. Collecting logs from chroot

5.6. Replacing klogd on Linux

5.7. A note on timezones and timestamps

5.8. Dropping messages

6. Reference

6.1. Source drivers

6.1.1. internal()

6.1.2. file()

6.1.3. pipe()

6.1.4. program()

6.1.5. sun-streams() driver

6.1.6. syslog()

6.1.7. tcp(), tcp6(), udp() and udp6()

6.1.8. unix-stream() and unix-dgram()

6.2. Destination drivers

6.2.1. file()

6.2.2. logstore()

6.2.3. pipe()

6.2.4. program()

6.2.5. sql()

6.2.6. syslog()

6.2.7. tcp(), tcp6(), udp(), and udp6()

6.2.8. unix-stream() & unix-dgram()

6.2.9. usertty()

6.3. Log path flags

6.4. Filter functions

6.4.1. Using regular expressions in filters

6.5. Macros

6.6. Message parsers

6.6.1. CSV parsers

6.6.2. Pattern databases

6.7. Rewriting messages

6.8. Regular expressions

6.9. Global options

6.10. TLS options

Appendix 1. The syslog-ng manual pages

syslog-ng — syslog-ng system logger application

syslog-ng.conf — syslog-ng configuration file

dqtool — Display the contents of a disk-buffer file created with syslog-ng Premium Edition

loggen — Generate syslog messages at a specified rate

lgstool — Inspect and validate the binary log files (logstores) created with syslog-ng Premium Edition

pdbtool — An application to test and convert syslog-ng pattern database rules

syslog-ng-ctl — Display message statistics and enable verbose, debug and trace modes in syslog-ng Premium Edition

Appendix 2. BalaBit syslog-ng Premium Edition License contract

2.1. SUBJECT OF THE LICENSE CONTRACT

2.2. DEFINITIONS

2.3. WORDS AND EXPRESSIONS

2.4. LICENSE GRANTS AND RESTRICTIONS

2.5. SUBSIDIARIES

2.6. INTELLECTUAL PROPERTY RIGHTS

2.7. TRADE MARKS

2.8. NEGLIGENT INFRINGEMENT

2.9. INTELLECTUAL PROPERTY INDEMNIFICATION

2.10. LICENSE FEE

2.11. WARRANTIES

2.12. DISCLAIMER OF WARRANTIES

2.13. LIMITATION OF LIABILITY

2.14. DURATION AND TERMINATION

2.15. AMENDMENTS

2.16. WAIVER

2.17. SEVERABILITY

2.18. NOTICES

2.19. MISCELLANEOUS

Appendix 3. Deprecated pattern database schemes

3.1. The syslog-ng pattern database format V1

3.2. The syslog-ng pattern database format V2

Appendix 4. Creative Commons Attribution Non-commercial No Derivatives (by-nc-nd) License

Glossary

List of syslog-ng PE parameters

Index