15.2.7. Procedure – Customizing the indexing of HTTP traffic

Purpose: 

To customize how PSM indexes HTTP traffic, complete the following steps.

Prerequisites: 

You can customize only the configuration of external indexers. The indexer running on the PSM host always uses the default HTTP configuration, which is the following:

{
  "General": {
    "Whitelist": ["text/.*", ".*json.*", "application/x-www-form-urlencoded", "multipart/.*"],
    "Blacklist": ["text/css", "application/javascript", "text/xslt", ".*xml.*"]
  },
  "Form": {
     "Blacklist": ["password", "pass"]
  },
  "Html": {
    "Attributes": ["href", "name", "value", "title", "id", "src"],
    "StrippedTags": ["script", "object", "style", "noscript", "embed", "video", "audio", "canvas", "svg"]
  }
}

Steps: 

  1. Create a configuration file for the HTTP indexer using a text editor. The configuration file uses the JSON format. For details on the configuration format, see Section 15.4, HTTP indexer configuration format.

    Note

    If you want to index HTTP POST messages, include the "application/x-www-form-urlencoded" Content-Type in the General > WhiteList list. The indexer will decode URL encoding (percentage encoding), and create key=value pairs from the form fields and their values. Note that in the values, the indexer will replace whitespace with the underscore (_) character. To avoid indexing sensitive information (for example, passwords from login forms), use the Form > Blacklist option.

  2. Copy the configuration file to the external hosts, to the /opt/external-indexer/usr/share/adp/httpconfig.json file.

  3. Reload the indexer service: systemctl restart external-indexer.service

  4. Repeat the above steps for your other external indexer hosts. Otherwise, it is possible that certain audit trails will be indexed using different indexer configuration.

  5. Disable the indexer that is running on the PSM host. Otherwise, it is possible that certain audit trails will be indexed using different indexer configuration.

    Navigate to Basic Settings > Local Services > Indexer service, and set the Maximum parallel audit trails to index on box option to 0.