The syslog-ng Agent for Windows 3.2 Administrator Guide

2.2.1. Procedure – Installing the syslog-ng Agent on the domain controller and the hosts of a domain

	Note
	Starting from version 3.0.3, the syslog-ng Agent sends only messages that are created after the agent has been installed. If you want to send old log messages to the syslog-ng server, download the Orca MSI editor here, open the .msi installer of the syslog-ng Agent, select Property, and change the value of the SENDOLDMESSAGES field to yes.

Steps: 

1. Download both the Microsoft Installer (.msi) version and the executable (.exe) version of the syslog-ng Agent installer to the domain controller host. Make sure to download the executable that includes the MMC snap-in module. Note that separate .msi intallers are available for 32-bit and 64-bit operating systems.

   	Note
   	Installing the syslog-ng Agent requires administrator privileges, but configuring the related group policies on the domain controller requires domain administrator or higher (for example enterprise administrator) privileges.

2. Install the syslog-ng Agent application to your domain controllers using the .exe installer.

   	Note
   	The syslog-ng Agent for Windows requires the Microsoft .NET Framework version 2.0. This package is usually already installed on most hosts. Download the package here.

3. Select Start > Control Panel > Administrative Tools > Active Directory Users and Computers, right-click on the Organizational Unit of the domain whose hosts you want to install the syslog-ng agent on, and select Properties.

4. Select Group Policy, and edit the Group Policy object you want to add the syslog-ng agent configuration to. Alternatively, you can create a new group policy object as well.

5. Select Computer Configuration, right-click on Software Settings, and select New > Package.

6. Navigate to the syslog-ng Agent for Windows .msi installer and select Open.

7. Select Assigned, then OK.

8. Select Computer Configuration > syslog-ng Agent Settings and configure the syslog-ng Agent. The members of the domain will use this configuration.

   

   Figure 2.2. syslog-ng Agent Settings

   
   

9. The syslog-ng Agent for Windows application will be automatically installed on the members of the domain when they are next rebooted. To perform the installation earlier, execute the gpupdate command on the members of the domain.

   	Note
   	If you do not want to install the syslog-ng Agent automatically from the domain controller, skip Steps 5-7, complete Step 8, then install the syslog-ng-agent-nosnapin-<versionnumber>-setup.exe file manually on the members of the domain. This method is useful if you do not want to install the syslog-ng Agent on every host of the domain.

3.2.1. Procedure – Configuring the destination logservers

Purpose: 

To configure a new destination, complete the following steps:

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. Select syslog-ng Agent Settings > Destinations, and double-click on Add new server.

3. Enter the hostname or the IP address of the logserver into the Server Name field. If your logserver is configured to accept messages on a non-standard port, type the port number into the Server Port field.

   

   Figure 3.2. Adding new server

   
   

4. On Messages tab, select the protocol used to transfer log messages and press Reset to apply the selected template. The following protocol templates are available:

   • Legacy BSD Syslog Protocol: Use the legacy BSD-syslog protocol specified in RFC3164. This option uses the following message template: <${PRI}>${BSDDATE} ${HOST} ${APP_NAME}[${PROCESS_ID}]: ${MSG}.

     

     Figure 3.3. Legacy BSD Syslog Protocol

     
     

     	Example 3.1. Legacy BSD Syslog Protocol log
     	<134>Oct 04 14:45:33 zts-win019.ztswin2008dom.balabit Microsoft-Windows-Eventlog[2880]: ZTSWIN2008DOM\balabit: System Microsoft-Windows-Eventlog: [Information] The Application log file was cleared. (EventID 104)

   • Syslog Protocol: Use the new IETF-syslog protocol specified in RFC 5424-5428. Starting from version 3.0, syslog-ng also supports the IETF-protocol.

     

     Figure 3.4. Syslog Protocol

     
     

     	Note
     	Starting with version 3.2, the syslog-ng Agent for Windows application uses the IETF-syslog protocol by default, and the default metadata to include is Macro names and values.

     When using the IETF-syslog protocol to transfer Eventlog messages, the syslog-ng Agent application can include the macros (name-value pairs) in the SDATA part of the log message. Select the data to add to the log messages from the Metadata to include list. The following options are available:

     • None: Include only the data mandated by RFC5424.

       [meta sequenceId="value" sysUpTime="value"] [origin ip="value" software="value"]

     • Macro names and values: Include every available Event macro, except EVENT_MSG (EVENT_MESSAGE), EVENT_MSG_XML (EVENT_MESSAGE_XML) and macros with null value.

       This is the default setting.

       499 <132>1 2010-09-28T12:02:30+02:00 zts-win004.ztswin2003dom.balabit testapp 1220 - [meta sequenceId="1" sysUpTime="1"][origin ip="zts-win004.ztswin2003dom.balabit" software="testapp"][win@18372.4 EVENT_ID="1000" EVENT_NAME="Application" EVENT_REC_NUM="1673" EVENT_SID="S-1-5-21-3460971693-970282485-2299281428-1001" EVENT_SID_TYPE="User" EVENT_SOURCE="testapp" EVENT_TYPE="Warning" EVENT_USERNAME="ZTS-WIN004\\balabit"] ZTS-WIN004\balabit: Application testapp: [Warning] test message (EventID 1000)

     • EventData (deprecated Agent v3.1 functionality): Include the name-vaule pairs from the EventData > <Data> field of the log message and the Keyword value from the System part of the XML Eventlog messages. Note that these fields are available only on Windows Vista and newer operating systems, and syslog-ng Agent 3.1. automatically included these values in the log messages.

       [EventData@18372.4 "name"="value"] [Keywords@18372.4 Keyword="value"]

       See the following sample log message:

       338 <132>1 2010-09-28T12:02:30+02:00 zts-win004.ztswin2003dom.balabit testapp 1620 - [meta sequenceId="1" sysUpTime="1"][origin ip="zts-win004.ztswin2003dom.balabit" software="testapp"][EventData@18372.4 Data="test message"][Keywords@18372.4 Keyword="Classic"] ZTS-WIN004\balabit: Application testapp: [Warning] test message (EventID 1000)332

   • Snare Protocol: Send log messages in a format compatible with the Snare log monitoring tool.

     

     Figure 3.5. Snare Protocol

     
     

     	Warning
     	3.1 and earlier versions of the syslog-ng Agent for Windows application used an imperfect message template to send Snare-compatible messages. This has been corrected in the Snare Protocol option. If you need the old, imperfect mode of operation, select a syslog protocol and select Message Type > Agent v3.1 Snare Compatible Message Type (deprecated), then click Reset Event Message Template.

     	Note
     	Snare is a tab-separated message format. Within the message part, agent replaces CRLF with 2 space, TAB character with 1 space. You cannot modify the log format if you have selected this protocol. >

     	Example 3.2. Snare log
     	<134>Oct 06 13:49:41 zts-win019.ztswin2008dom.balabit MSWinEventLog 1 Application 1 Wed Oct 06 13:49:41 2010 1 syslog-ng Agent S-1-5-21-551780264-1021859348-3425375765-1003 User Information zts-win019.ztswin2008dom.balabit None Application started 1

   	Note
   	Selecting the Syslog Protocol option is identical to using the syslog driver in the Linux/Unix version of syslog-ng. Similarly, selecting Legacy BSD Syslog Protocol is equivalent to the tcp driver of syslog-ng. Changing a protocol does not automatically change the protocol template used. To use a protocol-specific template, select Reset Protocol Template after modifying the protocol.

5. If needed, modify the template of the messages. The format of the messages can be different for the eventlog and the file sources.

6. If the host running syslog-ng Agent is sometimes logged in into a domain, sometimes not, then its hostname might change depending on its actual domain membership. This might cause that the hostname appearing in the syslog messages depends on the domain membership of the host. To avoid this situation, enable the Force DNS Hostname option. That way syslog-ng Agent resolves the name of its host from the DNS server, and uses the resolved FQDN in the syslog messages.

7. If you have a backup server that can accept log messages if the primary logserver becomes unavailable, select the Failover Servers tab, click Add, and enter the hostname or the IP address of the backup logserver into the Server Name field. Repeat this step if you have more than one backup servers.

   

   Figure 3.6. Failover Servers

   
   

8. If you want to send the log messages to more than on server in parallel, so that every server receives every message, repeat Steps 3-4 to add the secondary servers. Secondary servers may have failover servers as well.

   	Note
   	The syslog-ng Agent for Windows application considers a message received by the logserver if the primary server of the destination, or one of its failover servers receives it. To modify which server of a destination is the primary server, select syslog-ng Agent Settings > Destinations, right-click on the server you want to be primary, select Properties > Set Primary Server.

9. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

3.2.2. Procedure – Limiting the rate of messages

Purpose: 

The syslog-ng Agent can control the rate of messages (message per second) sent to the central server. That way sudden message-bursts can be avoided, and the load of the server is decreased.

To limit the number of messages sent to a destination, complete the following steps:

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. Select syslog-ng Agent Settings > Destinations.

3. Select the destination server and select Properties. To limit the number of messages that the syslog-ng agent sends to the server per second, enter the desired limit into the Throttle field. By default (0), the syslog-ng agent does not limit the number of messages sent.

   Note

   The throttling parameter applies to the total number of messages sent, not to every source independently. The same value applies to the failover servers of the destination. If you are sending messages to multiple servers, then the speed of the primary server is important: if the primary server cannot accept the messages fast enough, the syslog-ng agent will reduce the number of sent messages to match the speed of the primary server, even if the secondary servers could accept messages faster. If the secondary servers cannot accept messages as fast as the primary server, then the secondary servers will lose messages; the syslog-ng agent will not slow down to wait for them.

4. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

3.3.2. Procedure – Managing file sources

Purpose: 

The syslog-ng Agent for Windows application can collect log messages from text files. It can process messages spanning multiple lines, and supports the use of wildcards (*, ?) in filenames to be able to follow log files that are automatically rotated. To configure file sources, complete the following steps:

Figure 3.8. Managing file sources

Figure 3.9. Sources properties

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. Select syslog-ng Agent Settings > File Sources, double-click on Sources, and check the Enable option.

3. Select Add > Browse, and select the folder containing the log files in the Base Directory field. Select or enter the name and extension of the log files in the File Name Filter field. Wildcards may be used. The syslog-ng agent will forward log messages from every file that is located in this folder and has a name that matches the filter expression.

   	Tip
   	When specifying the Base Directory, you can use the environment variables of Windows, for example %WINDIR%, %SYSTEMROOT%, %PROGRAMFILES%, and so on.

   	Warning
   	Note that when managing members of a domain, the selected path must be available on the domain members, for example C:\logs must be available on the client hosts and not on the domain controller.

4. • To send messages from the files located in the subfolders of the folder set as Base Directory, select the Recursive option.

   • To send messages only from the file that was last modified, select the Last Modified File Only option.

     Note

     When using the Last Modified File Only option with a file source that has wildcard in the filename (for example *.log), the following will happen. When started for the first time, the agent will send the contents of every matching file to the central server, and store the position of the last message in the file with the most recent modification date. When new messages are written to this file, the agent will send only the new messages. However, if an older file is modified, the agent will resend the entire contents of this newly modified file, and store the position of the last message in this file only. When you use wildcards together with the Last Modified File Only option, make sure that older files will not be modified.

     If you are forwarding the logs of Internet Information Server (IIS) 5 applications, select the IIS 5.x Log option.

     	Note
     	If this option is not selected, the syslog-ng Agent monitors every matching file in the folder for changes, and sends new log messages from all files.

   • To send messages only from the file that was last modified of every subfolder of the Base Directory, select both the Last Modified File Only and the Recursive options.

   • To change the log facility or the log severity associated to the file source, select the desired facility or priority from the Log Facility or Log Severity fields, respectively.

     Note

     Significant changes to the settings of a file source may cause the syslog-ng Agent to resend the entire contents of the matching files. This means that log messages already sent earlier to the syslog-ng server may be resent and thus duplicated in the server logs. Configuration changes that may result in such behavior are: changing the Base Directory, changing filter options, changing recursivity and Last Modified File Only options.

5. By default, the syslog-ng Agent application assumes that the source files are encoded using the default windows ANSI code page, specific to the locale of the host. If the files have a different encoding, select it from the File Encoding field. Unicode, UTF-16, and UTF-32 encodings are currently not supported. Note that the log messages are sent to the destinations using UTF-8 encoding.

6. If a log messages in the log file consists of multiple lines, that is, the log messages contain newline characters, configure syslog-ng Agent to process the related lines as a single message.

   The syslog-ng Agent application can automatically handle Apache Tomcat Catalina and Oracle SQL log messages. To process such messages, select the name of the application from the Multiple Lines > Application field. Note that the timestamp of Tomcat log messages depends on the locale of the host.

   To process multi-line log messages of a different application, complete the following steps.

   1. Select Multiple Lines > Application > Custom, and set the Multiple Lines > Prefix and optionally the Multiple Lines > Garbage fields.

   2. Specify a string or regular expression that matches the beginning of the log messages in the Multiple Lines > Prefix field. If the Prefix option is set, the syslog-ng Agent ignores newline characters from the source until a line matches the regular expression again, and treats the lines between the matching lines as a single message.

      	Note
      	Use as simple regular expressions as possible, because complex regular expressions can severely reduce the rate of processing multi-line messages.

   3. Use the Multiple Lines > Garbage option when processing multi-line messages that contain unneeded parts between the messages. Specify a string or regular expression that matches the beginning of the unneeded message parts. If the Garbage option is set, the syslog-ng Agent ignores lines between the line matching the Garbage expression and the next line matching Prefix.

      When receiving multi-line messages from a source when the Garbage option is set but no matching line is received between two lines that match Prefix, the syslog-ng Agent application will continue to process the incoming lines as a single message until a line matching Garbage is received.

      	Warning
      	If the Garbage option is set, the syslog-ng Agent application discards lines between the line matching the Garbage and the next line matching Prefix expressions.

   4. Optional Step: After creating and testing a custom pattern, please consider sending your pattern to BalaBit so we can include it in a future version of syslog-ng Agent. To share your pattern with BalaBit and other syslog-ng Agent users, click Multiple Lines > Send custom pattern to BalaBit. Your e-mail application will open, with an e-mail containing the application name and the pattern.

7. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

3.3.3. Procedure – Configuring global settings

Purpose: 

The syslog-ng Agent for Windows application has some global settings that can apply to both eventlog and file sources. To configure the global settings, complete the following procedure:

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. Select syslog-ng Agent Settings and double-click on Global Settings.

3. Set the default log facility associated to the messages.

4. By default, the filters and regular expressions (see Section 3.5, Filtering messages) used in the message filters are case-sensitive. To make them case-insensitive, select the Regular Expressions Ignore Case or the Filters Ignore Case options, or both.

   	Note
   	The Regular Expressions Ignore Case option makes the Message Contents filter case-insensitive for both file and eventlog sources. The Filters Ignore Case option makes the Computers, Sources and Categories, and the Users filter case-insensitive.

5. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

3.3.4. Procedure – Disabling sources and filters globally

Purpose: 

Filters and sources can be disabled globally as well. Disabling filters or sources means that the syslog-ng agent ignores the disabled settings: that is, if the file sources are disabled, the agent does not send the messages from the files to the server. For details, see the following procedure.

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. • To disable eventlog sources, select syslog-ng Agent Settings, right-click on Eventlog Sources, then select Properties > Disable.

   • To disable file sources, select syslog-ng Agent Settings, right-click on File Sources, then select Properties > Disable.

   • To disable eventlog filters, select syslog-ng Agent Settings > Destinations, right-click on Global Event Filters, then select Properties > Disable.

   • To disable file filters, select syslog-ng Agent Settings > Destinations, right-click on Global File Filters, then select Properties > Disable.

3. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

3.4.1. Procedure – Enabling encrypted connections

Purpose: 

To enable SSL-encrypted connections to the server, complete the following steps:

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. Select syslog-ng Agent Settings > Destinations.

3. Right-click on the server that accepts encrypted connections and select Properties.

4. Select the Use SSL option.

   	Warning
   	The connection can be established only if the Certificate Authority (CA) that issued the certificate of the server is available in the Certificate Store (MMC > Certificates > Computer Account > Local Computer > Trusted Root Certificates) of the Windows-based host. For details on importing certificates, see Procedure 3.4.3, Importing certificates with the Microsoft Management Console.

   Note

   The subject_alt_name parameter (or the Common Name parameter if the subject_alt_name parameter is empty) of the server's certificate must contain the hostname or the IP address (as resolved from the syslog-ng clients and relays) of the server (for example syslog-ng.example.com). Alternatively, the Common Name or the subject_alt_name parameter can contain a generic hostname, for example *.example.com. Note that if the Common Name of the certificate contains a generic hostname, do not specify a specific hostname or an IP address in the subject_alt_name parameter.

5. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

3.4.3. Procedure – Importing certificates with the Microsoft Management Console

Purpose: 

To import a certificate, complete the following steps.

Steps: 

1. Start Microsoft Management Console by executing mmc.exe (Start menu Run application).

   	Note
   	Running mmc.exe requires administrator privileges.

2. Click on the Add/Remove snap-in item of the File menu.

3. Click Add, select the Certificates module, and click Add.

4. Select Computer account in the displayed window and click Next.

5. Select Local computer and click Close.

6. To import the certificate of the syslog-ng server, navigate to Console Root > Certificates > Trusted Root Certificate Authorities > Certificates.

   To import a certificate for the syslog-ng agent to perform mutual authentication, navigate to Console Root > Certificates > Personal > Certificates.

7. Right-click on the Certificates folder and from the appearing menu select All tasks > Import. The Certificate Import Wizard will be displayed. Click Next.

   Optional step: Certificates used to authenticate the syslog-ng agent in mutual authentication include the private key. Provide the password for the private key when requested.

8. Windows offers a suitable certificate store by default, so click Next.

9. Click Finish on the summary window and Yes on the window that marks the successful importing of the certificate.

3.5.1. Procedure – Filtering eventlog messages

Purpose: 

The following types of filters are available for eventlog sources:

To modify the filters used for eventlog messages, complete the following procedure:

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. • To apply filters globally to every eventlog message, select syslog-ng Agent Settings > Destinations > Global Event Filters, and right-click Global Event Filters.

   • To apply filters only to a specific destination, select syslog-ng Agent Settings > Destinations, select the destination server, then select Event Filters. Right-click Event Filters.

   	Note
   	If you want to use both global and local (server side) filtering, first global filters will be applied to the eventlog messages and then the local filters.

3. Select Properties > Enable > OK.

   

   Figure 3.10. Global event filters

   
   

4. To use whitelist-filtering, select White List Filtering. By default, syslog-ng Agent uses blacklist filtering.

5. On the right-hand pane, double-click on the type of filter you want to create.

6. • To ignore messages sent by a specific application, or messages of the application with a specific event ID, double-click on Sources and Event ID, select Add, and select the name of the source (application) whose messages you want to ignore from the Source Name field. To ignore only specific messages of the application, enter the ID of the event into the Event ID field. Select Add > Apply.

     

     Figure 3.11. Sources and Event ID

     
     

   • To ignore messages that contain a specific string or text, double-click on Message Contents, enter the search term or a regular expression into the Regular Expression field, then select Add > Apply.

     

     Figure 3.12. Message Contents

     
     

   • To ignore messages sent by a specific application, or messages of the application that fall into a specific category, double-click on Sources and Categories, select Add, and select the name of the application whose messages you want to ignore from the Application Name field. To ignore only those messages of the application that fall into a specific category, enter the name of the category into the Category field. Select Add > Apply.

   • To ignore messages sent by a specific user, double-click on Users, enter the name of the user into the User field, then select Add > Apply.

   • To ignore messages sent by a specific computer (host), double-click on Computers, enter the name of the user into the Computer field, then select Add > Apply.

   • Event Types: To ignore messages of a specific event-type, double-click on Event Types, select the event types to ignore, and select Ok > Apply.

     Note

     Under Windows Vista and Server 2008, Windows labels certain messages as level 3 and the Event Viewer labels such messages as warnings. This is against the official specification: level 3 should not be used; and only level 2 messages are warnings. To filter these events, you have to manually add a new event type to the registry and set its value to 3, for example HKEY_LOCAL_MACHINE\SOFTWARE\BalaBit\syslog-ng Agent\Local Settings\EventSources\Filter\Type\Rule0\Type=3

7. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

3.5.2. Procedure – Filtering file messages

Purpose: 

The following types of filters are available for file sources:

To modify the filters used for file messages, complete the following procedure:

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. • To apply filters globally to every file message, select syslog-ng Agent Settings > Destinations > Global File Filters, and right-click Global File Filters.

   • To apply filters only to a specific destination, select syslog-ng Agent Settings > Destinations, select the destination server, then select File Filters. Right-click File Filters.

   	Note
   	If you want to use both global and local (server side) filtering, first global filters will be applied to the file messages and then the local filters.

3. Select Properties > Enable > OK.

   

   Figure 3.13. Global file filters

   
   

4. To use whitelist-filtering, select White List Filtering. By default, syslog-ng Agent uses blacklist filtering.

5. On the right-hand pane, double-click on the type of filter you want to create.

6. • To ignore messages that contain a specific string or text, double-click on Message Contents, enter the search term or a regular expression into the Regular Expression field, then select Add.

7. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

3.6.1. Procedure – Customizing messages using templates

Purpose: 

To create a template, complete the following procedure:

	Warning
	These macros are available only in the syslog-ng Agent for Windows. To recognize Windows-specific elements of the log message (for example eventlog-related macros) on the syslog-ng server, you have to use parsers on the syslog-ng server. The parser must be configured to match the message format set in the syslog-ng Agent.

Steps: 

1. Start the configuration interface of the syslog-ng Agent for Windows application.

2. Select syslog-ng Agent Settings > Destinations. Select your logserver, and click Properties.

3. To change the format of messages received from eventlog sources, type the message format you want to use into the Event Message Format > Message Template field.

   To change the format of messages received from file sources, type the message format you want to use into the File Message Format > Message Template field.

   Do not forget to add the $ character before macros. For a complete list of the available macros, see Section 3.6.3, Macros available in the syslog-ng Agent.

   For example, to send the messages in the DATE HOSTNAME MESSAGE format, type Date:$DATE Hostname:$HOST Logmessage:$MESSAGE.

   Note that the $MESSAGE macro contains not only the text of the log message, but also additional information received from the message source, such as the name of the eventlog container, or the file, as set in the eventlog-specific and file-specific templates.

   	Note
   	Templates are assigned to a single destination server, so it is possible to use different templates for different servers. However, a server and its failover servers always receive the same message.

   Warning

   If you have more than one destination servers configured (separate servers, not in failover mode), and you want to use the same template for every server, you must manually copy the template into the configuration of each server. Template modifications are not applied automatically to every server. To use a Snare-compatible message format, select Message Type > Snare Compatible Message Type and click Reset Message Template.

4. Click OK.

5. To activate the changes, restart the syslog-ng Agent service.