Home › Products › syslog-ng Store Box ›
syslog-ng Store Box

Classification

Take the Next Step

Even though SSB is not a log analyzing engine, it is able to classify individual log messages using artificial ignorance, much like the popular logcheck application of the Unix world. SSB comes with a built-in database of log message patterns that are considered "normal". Messages matching these patterns are produced during the legitimate use of the applications (e.g., sendmail, Postfix, MySQL, etc.), and are unimportant from the log monitoring perspective, while the remaining messages may contain something "interesting". The administrators can define log patterns on the SSB interface, label matching messages (e.g., security event, etc.) and request alerts if a specific pattern is encountered. For thorough log analysis, SSB can also forward the incoming log messages to external log analyzing engines.

Real-time log message classification

The syslog-ng application can compare the contents of the log messages to a database of predefined message patterns. By comparing the messages to the known patterns, syslog-ng is able to identify the exact type of the messages, and sort them into message classes. The message classes can be used to classify the type of the event described in the log message. The message classes can be customized, and for example can label the messages as user login, applic- ation crash, file transfer, etc. events.

sample log message

The functionality of the pattern database is similar to that of the logcheck project, but the syslog-ng aproach has the following advantages:

  • The syslog-ng patterns are much easier to write and maintain than the regular expressions used by logcheck.
  • It is much easier to understand syslog-ng pattens than regular expressions.
  • Pattern matching based on regular expressions is computationally very intensive, especially when the number of patterns increases. The solution used by syslog-ng can be performed real-time, and is independent from the number of patterns, so it scales much better.

To find the pattern that matches a particular message, syslog-ng uses a method called longest prefix match radix tree. This means that syslog-ng creates a tree structure of the available patterns, where the different characters available in the patterns for a given position are the branches of the tree.

To make the message classification flexible and robust, the patterns can contain pattern parsers: elements that match on a set of characters. For example, the NUMBER parser matches on any integer numbers (e.g., 1, 123, 894054, etc.). Other pattern parsers match on various strings and IP addresses.

The following patterns describe the same message:

Accepted password for bazsi from 10.50.0.247 port 42156 ssh2

A regular expression matching this message from the logcheck project:
Accepted \ (gssapi(-with-mic|-keyex)?|rsa|dsa|password|publickey|keyboard-interactive/pam) \ for [^[:space:]]+ from [^[:space:]]+ port [0-9]+( (ssh|ssh2))?

A syslog-ng database pattern for this message:
Accepted @QSTRING:auth_method: @ for@QSTRING:username: @from\ @QSTRING:client_addr: @port @NUMBER:port:@ ssh2

Learn more about the product feature areas as below

Back to top Or Back to the features