Installation and Configuration

Prerequisites

This app depends on data models included in the Splunk Common Information Model Add-on, specifically the Web data model. Please review the information on configuring the acceleration on the data model.

The Splunk Common Information Model Add-on can be downloaded from Splunkbase.

This app has been tested with versions 4.X of the CIM add-on.

In order to make the app respond and load quickly, accelerated data models are used to provide summary data. For this data to be available, the Web data model must be accelerated. Information on how to enable acceleration for the Web data model can be found here. The data model must be accelerated for the length of time for which you would like to see reporting.

The Breakdown of Sourcetypes in Indexes in Dataset panel on the Data Transparency Overview dashboard will only be seen if the Sankey Diagram visualization is installed.

The Sankey Diagram Visualization can be downloaded from Sankey Diagram Custom Visualization.

User-agent strings are very difficult to parse efficiently using regular expressions. To make this easier and faster, an external lookup is used. By default, the TA-user-agents is used, as it requires no additional data download or internet access.

TA-user-agents can be downloaded from here: https://splunkbase.splunk.com/app/1843/

The user-agent lookup being used can be customized. See the section titled Customizing user-agent lookups for more information.

Not all organizations are the same, so there is a good chance that the default bad categories will need to be customized to meet the needs and policies of your organization. Additionally, customizations may be required based on the type of web proxy device generating the events.

Different web proxy vendors is different category names, so one list wouldn’t cover all vendors. To enable support for different vendors, a macro named apl_web_wfa_lookup is used.

Currently, this app comes pre-configured with lookups for 7 different products:

Barracuda: apl_web_barracuda_wfa
Bluecoat (default): apl_web_bluecoat_wfa
Cisco WSA: apl_web_cisco_wsa_wfa
McAfee Web Gateway: apl_web_mcafee_wg_wfa
Palo Alto Firewalls: apl_web_palo_alto_wfa
URLBlacklist: apl_web_urlblacklist_wfa
Websense: apl_web_websense_wfa

These lookup files can be found in $SPLUNK_HOME/etc/apps/aplura_web_proxies_app/lookups.

To configure the app to use the correct lookup for your data, edit the apl_web_wfa_lookup macro to match the vendor from the list above. More information on editing Splunk macros can be found in the Splunk documentation.

<div class="note"> While Splunk has configuration layering for configuration files, it does not have the same layering for lookup files. This means that if you customize one of these lookup files instead of creating your own, you need to make sure to back it up before upgrading this app, as it may be overwritten. </div>

Because your organizations policies may differ from what is represented in the provided lookups, or you may be using a product which is not listed above, you may wish to use your own custom lookup for Policy Exceptions. A custom lookup for Policy Exceptions should have two columns, category and wfa. The category column should match the name of the category produced by your proxy device to which visits from clients are not allowed, or need to be reviewed. The second column, wfa, should always have a value of TRUE. You can always look at the existing lookups for an example.

Once you have created your lookup, edit the apl_web_wfa_lookup macro with the name of the lookup you have created.

The use of wildcards for the lookup file is not configured out of the box, however, it can by copying the transforms.conf file from the /default directory to the /local directory and adjusting the configuration of the apl_web_wfa_lookup macro in macros.conf.

By default, this app uses an external lookup to provide analytics on user-agent string. The default lookup used is the TA-user-agents lookup, however, this can be customized to use another lookup which may be more accurate or provide more fields for customizing the dashboard.

To customize the lookup, download and installed the other lookup. Copy the macros.conf file from the aplura_web_proxies_app/default directory of the app to a aplura_web_proxies_app/local directory (which you may need to create). Edit the macros.conf file, and edit the apl_web_user_agent_lookup macro to be the name of the custom lookup, and then edit the apl_web_ua_family_field to match the high-level product name (for example IE or Firefox). The use of the macros expect that the field being used for the lookup will be the CIM-compliant http_user_agent. If the lookup is expecting something else, then the dashboard will need to be customized to match.

References and Downloads

Sankey Diagram Custom Visualization

Software requirements

Splunk Enterprise system requirements

Because this App runs on Splunk Enterprise, all the Splunk Enterprise system requirements apply.

Download

Download PAVO Web Proxies App for Splunk at https://splunkbase.splunk.com/app/4231.

Installation steps

This app should be installed on a search head where the Web data model has been accelerated. More information on installing or upgrading Splunk apps can be found here.

Installation Process Overview

Make sure the field extractions and tags on your proxy events are correct.
Install the Splunk Common Information Model Add-on (skip if you are installing on an ES search head).
Install the PAVO Web Proxies App for Splunk for Splunk.
Enable accelerations on the Web data model (skip if you are installing on an ES search head).
Wait for the accelerations to start. After the acceleration searches have run, you should start seeing the dashboards populate.

Deploy to single server instance

Follow these steps to install the app in a single server instance of Splunk Enterprise:

Deploy as you would any App, and restart Splunk.
Configure.

Deploy to Splunk Cloud

Have your Splunk Cloud Support handle this installation.

Deploy to a Distributed Environment

For each Search Head in the environment, deploy a copy of the App.

PAVO Web Proxies App for Splunk

0