The Verizon Autonomous Threat Hunting App for Splunk provides an integration between Splunk and the Verizon Autonomous Threat Hunting service. It implements dashboards to visualize the events ingested using the Verizon Autonomous Threat Hunting Alerts Add-on. It also allows the collection and submission of statistical summaries extracted from the Splunk log data to the Verizon Autonomous Threat Hunting engine.
The Verizon Autonomous Threat Hunting engine uses groundbreaking machine learning and data science techniques, cyber threat intelligence indicators from a variety of sources, and the historical patterns of communications of each organization's network to determine a very small subset of potentially infected or compromised machines. This technology is known for generating very accurate results, and is able to extrapolate from existing threat data to detect novel threats before other existing security products.
Upon receiving those suspicious communications, organizations' information security monitoring, digital forensics and/or incident response teams can then focus on the analysis of the small group of internal machines that originated them.
You are not allowed to use this App unless you have an existing Service or Trial contract with Verizon for the use of this product Verizon Autonomous Threat Hunting and have reviewed and agreed to all the contractual terms of that contract.
This application has the following requirements to work properly:
Splunk 6.5 or above;
CIM-compliant log data from perimeter firewalls, web proxy or URL filtering solutions indexed in Splunk that allow traffic originating from your endpoints towards the outside world to be analyzed. It is also critical that the CIM fields be visible on a system level for this to work;
The list of all the public IPv4 ranges owned by your organization, such as those used in DMZs;
The list of all DNS domains used by your organization, including internal domains used in Active Directory and internal web applications;
If you are a Splunk Enterprise user, ensure the
maxchars entry of the
kv stanza of limits.conf is equal to or greater than 102400. If you are a Splunk Cloud customer, please contact support to ensure this is properly configured on your cluster.
If you are a Splunk Enterprise user, ensure the
maxresultrows entry of the
searchresults stanza of limits.conf is equal to and NOT less than 50000. This limit should not exceed 50000. Setting this limit higher than 50000 causes instability.
After ensuring that all the above requirements are met, you should install the app according to the Splunk documentation.
The first thing that needs to be set up is credentials and connectivity to the Verizon Autonomous Threat Hunting API. This setup page will need to read and write secrets to the Splunk storage passwords functionality for security reasons. Due to Splunk requirements, this means this setup page can only be fully utilized by user accounts with the
API Key should be set to a valid Verizon Autonomous Threat Hunting API key.
Proxy Server Address and
Proxy Server Port should be filled respectively with an IP address or hostname and a TCP port if an HTTPS proxy should be used to contact api.niddel.com from the Search Head were this app was installed. Otherwise, leave these fields blank.
Password should be filled only if a proxy is in use and it requires authentication. Otherwise, leave these fields blank.
When the Save button is clicked, the app will test its connectivity and attempt to use the provided API key and/or proxy settings to contact the Verizon Autonomous Threat Hunting API. The settings will not be saved unless this test succeeds.
Once the credentials have been set up, you can configure the app to collect statistical summaries based on the log data indexed into Splunk and submit them to the Verizon Autonomous Threat Hunting engine for analysis.
Organizationselects the organization or sub-organization for which you want to submit logs to the Verizon Autonomous Threat Hunting engine. If the configured API key has access to multiple organizations, since the app version 2.1.5, multiple organizations can submit logs once the settings and saved searches are stored separately within the Splunk instance.
Since the app version 2.1.5, the network topology settings is collected from the Verizon Autonomous Threat Hunting and saved to the local Splunk deployment settings.
Public IPv4 ranges must contain a list of public subnets that are owned by your organization, in CIDR notation. It is critical that all networks with valid IP address ranges, such as DMZs, be listed here so that the app can correctly separate outbound traffic from inbound and internal traffic.
Public and internal DNS domains must contain a list of DNS domains that are ownned by your organization. It is critical that all domains used publicly and internally are included. For example, domains used in Active Directory, internal web systems, public e-mail addresses and public websites should be listed.
Countries defines the countries present in the organization network topology. This information is highly relevant for the Verizon Autonomous Threat Hunting engine learns about the geographical context of outbound traffic against whitelists and threat intelligence indicators of compromise.
Recursive DNS Servers,
Web Proxy servers,
Mail Servers and
Exclude Public IPv4 ranges provides more clarity to the Verizon Autonomous Threat Hunting on roles of internal network assets owned by your organization and helps prevent the generation of false positives alerts during the automated threat hunting detection.
Before the app version 2.1.5, the information for
Public IPv4 ranges and
Public and internal DNS domains were configured locally in the Splunk deployment. If you are upgrading to the version 2.1.5 or newer, a similar screen to the screenshot below will be presented in order to compare and review the network topology stored by the older version and new settings collected from the Verizon Autonomous Threat Hunting engine.
Once you have compared and revised the network topology settings, you can click on
Delete Local Settings button and the following warning will be presented as a confirmation prior to the deletion.
Perform reverse DNS lookup of source IPs controls whether this app attempts to resolve the hostnames of the internal endpoints seen in the log data when necessary. See
Reverse DNS Resolution below for more details.
Collect user IDs controls whether this app attempts to collect the user IDs present in the log data (as per the
user CIM field).
Collect firewall logs must be checked if you wish firewall logs to be collected by the app and uploaded to the Verizon Autonomous Threat Hunting engine for processing. If you have established alternative ways of submitting your firewall logs or wish to use this app just to view the results of the analysis, uncheck this. Otherwise, leave it checked.
Firewall logs search must contain a valid Splunk search query that selects the existing perimeter firewall log data. It is important that the firewall log data be normalized according to what is described in the CIM Network Traffic data model.
Collect web proxy logs must be checked if you wish web proxy logs to be collected by the app and uploaded to the Verizon Autonomous Threat Hunting engine for processing. If you have established alternative ways of submitting your web proxy logs or wish to use this app just to view the results of the analysis, uncheck this. Otherwise, leave it checked.
Web proxy logs search must contain a valid Splunk search query that selects the existing IDS log data. It is important that the IDS log data be normalized according to what is described in the CIM Web proxy data model.
DNS log collection is supported since Niddel Splunk App version 2.1.4.
Collect DNS logs logs must be checked if you wish DNS logs to be collected by the app and uploaded to the Verizon Autonomous Threat Hunting engine for processing. If you have established alternative ways of submitting your DNS logs or wish to use this app just to view the results of the analysis, uncheck this. Otherwise, leave it checked.
DNS logs search must contain a valid Splunk search query that selects the existing DNS log data. It is important that the DNS log data be normalized according to what is described in the CIM DNS data model.
This app can optionally use the built-in reverse DNS lookup Splunk functionality to try to find out the hostname of each internal machine observed on the firewall logs. This will only be done if the machine hostnames are not present on the logs already and stored on
src CIM fields.
Having the hostname can greatly improve your team's ability to investigate suspicious communications and we highly recommend to enable this if possible. See
Log Collection Setup above for details on how to enable it. This is usually not recommended for Splunk Cloud customers, since their search heads will not typically be using their on-premises corporate DNS servers for name resolution.
Great effort was taken to avoid resolving each machine more than once each hour to reduce any performance impact. Still, please monitor the performance of your Splunk cluster and of the recursive DNS server it uses after enabling this.
This app will generate detailed logs of its activity to
$SPLUNK_HOME/var/log/splunk/niddel2.log. By default Splunk indexes this into the
_internal index, which allows the following search to be used to view its contents:
If you have problems, make sure you have a valid API key and that a proxy is correctly configured if needed. Also, ensure your proxy is not blocking access to the Verizon Autonomous Threat Hunting API HTTPS endpoint at
Some customers have reported SSL errors on the communication with the Verizon Autonomous Threat Hunting API, which will include the following log line:
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:676)
This error can be caused by proxies of next-generation firewalls that perform SSL inspection of HTTPS traffic. The solution in this case is to configure appropriate exceptions:
* https://api.niddel.com for Splunk cluster members running both the add-on and the app; * AWS S3 API endpoints for Splunk cluster members running the app.
Finally, you can always open a support ticket by sending an e-mail message to firstname.lastname@example.org.
Verizon Autonomous Threat Hunting App for Splunk
Copyright 2018, Verizon.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
* DNS log collection optmization
* Updated 'magnetsdk2' to version 1.6.6
Splunk AppInspect evaluates Splunk apps against a set of Splunk-defined criteria to assess the validity and security of an app package and components.
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.