RiskIQ Add-on for Splunk

OVERVIEW

The RiskIQ Add-on for Splunk is used to get data from the RiskIQ platform. For the dashboard with RiskIQ data, please install RiskIQ App for Splunk.

• Author - RiskIQ
• Version - 1.1.0
• Build - 18
• Creates Index - False
• Prerequisites - RiskIQ API Key, Token, Customer Name for data collection
• Compatible with:
• Splunk Enterprise version: 7.2, 7.3
• OS: Platform independent

RELEASE NOTES

Version 1.1.0

• Moved and Added lookups and custom commands in TA from main app
• Updated /event/search API data collection to use parameter scroll instead of offset to resolve the Internal Server Error
• Added proxy support
• Bifurcated assets and events data into different source types
• Provide support for enabling/disabling data inputs
• Provided support to collect only new and updated assets information
• Added validation for API credentials in setup page

OPEN SOURCE COMPONENTS AND LICENSES

• Some of the components included in RiskIQ Add-on for Splunk are licensed under free or open source licenses. We wish to thank the contributors to those projects.

  • requests version 2.20.1 http://docs.python-requests.org/en/master/ (LICENSE https://github.com/requests/requests/blob/master/LICENSE)

  • certifi version 2017.04.17 https://pypi.python.org/pypi/certifi (LICENSE https://github.com/certifi/python-certifi/blob/master/LICENSE)

  • chardet version 3.0.4 https://pypi.python.org/pypi/chardet (LICENSE https://github.com/chardet/chardet/blob/master/LICENSE)

  • idna version 2.5 https://pypi.python.org/pypi/idna/2.5 (LICENSE https://github.com/kjd/idna/blob/master/LICENSE.rst)

  • urllib3 version 1.21.1 https://pypi.python.org/pypi/urllib3/1.21.1 (LICENSE https://github.com/shazow/urllib3/blob/master/LICENSE.txt)

  • pytz version 2019.3 http://pythonhosted.org/pytz (LICENSE https://pythonhosted.org/pytz/#license)

THIRD PARTY LIBARIES

• We have used external library iso8601(version: 0.1.12) to convert time in ISO 8601 format to epoch format.
  https://pypi.python.org/pypi/iso8601

TOPOLOGY AND SETTING UP SPLUNK ENVIRONMENT

This app has been distributed in two parts.
1. RiskIQ Add-on for Splunk, which collects data from RiskIQ API.
2. RiskIQ App for Splunk helps in visualizing and analyzing RiskIQ data.

This app can be set up in two ways:
1. Standalone Mode:
* Install both the RiskIQ App for Splunk and RiskIQ Add-on for Splunk.
2. Clustered Environment:
* Install the RiskIQ App for Splunk and RiskIQ Add-on for Splunk on search head.
* To setup RiskIQ Add-on for Splunk on the search head follow below setps:
1. On Search head deployer, Go to $SPLUNK_HOME$/etc/shcluster/apps/TA-RiskIQ/local.
2. Create app.conf file and add below stanza.
[install] is_configured = 1
3. Save the file and push the bundle to search head.
* Install the RiskIQ Add-on for Splunk on heavy forwarder.
* User needs to create riqsummary index on the indexer.
* User needs to set up Add-on to start data collection on heavy forwarder.

External Data Sources

• We are using RiskIQ API for data collection

INSTALLATION

Follow the below-listed steps to install an Add-on from the bundle:

• Download the App package.
• From the UI navigate to Apps->Manage Apps.
• In the top right corner select Install app from file.
• Select Choose File and select the App package.
• Select Upload and follow the prompts.

UPGRADE

Follow the below steps when upgrading from TA-RiskIQ 1.0.0 to 1.1.0

• From the UI navigate to Apps->Manage Apps.
• In the top right corner select Install app from file.
• Select Choose File and select the App package.
• Check the upgrade option.
• Select Upload and follow the prompts.
• Restart Splunk.
• Navigate to Settings->Data Inputs->Scripts and disable the AccessRiskIQ.py.
• Enable riskiq_events.py and riskiq_assets.py to collect data of events and assets respectively.

CONFIGURATION

• After the installation, you'll be asked to restart the Splunk. Click on Restart Now.
• After the restart, you need to set up the data collection. From the UI navigate to Apps->Manage Apps->RiskIQ Add-on for Splunk->Set up.
• Enter the following details of your RiskIQ Instance and save it.
• API Token: RiskIQ API token
• Key: RiskIQ API key
• Customer: Customer name
• Page Size: Number of results to fetch in single API call. Defaults to 500
• Enable assets data collection: Check this to enable input and collect assets data
• Enable events data collection: Check this to enable input and collect events data
• Collect only new and changed assets: If this option is checked, only the new and changed assets will be collected. By default all assets are collected once in a day. To create/use default visuals displaying trend data, ingest all asset data every day by unchecking this option. To get only updated data then select this option.
• Latest Updated Time :This option is shown when we check collect only new and changed assets. Provide the time in PST Time Zone with format(YYYY-MM-DDTHH:MM:SS.sss)
• Enable Proxy: Check this option if you have proxy configured in your environment.
• Proxy Scheme: Provide proxy protocol (http/https/socks4/socks5)
• IP / Hostname: IP address or hostname of proxy instance.
• Port: Port used to connect to proxy instance.
• Require Authentication for Proxy: Check this option if your proxy configuration requires authentication.
• Username: Provide proxy username

• Password: Provide proxy password

• By default, SSL Verification will be true. If you don't want to verify your SSL certificate, follow steps:

• Go to $SPLUNK_HOME$/etc/apps/TA-RiskIQ/local.
• Open appsetup.conf file and find the stanza app_config.
• Update value of verify_ssl from true to false.
• Save the file and restart Splunk.
  TA RiskIQ is configured and ready to be used.

CUSTOM COMMANDS

• getcves - This command is used to get CVE(Common Vulnerabilities and Exposures) data from https://cve.mitre.org/data/downloads/allitems.csv.

ADDING SSL CERTIFICATE

If you have used SSL certificate for your domain, you need to add certificate into Splunk. Follow below-listed steps to do the same:
Go to $SPLUNK_HOME$/etc/apps/TA-RiskIQ/bin/lib/certifi.
Open cacert.pem file and add your custom certificate details at the end of the file.
* Save the file and restart the Splunk.

Note: If your vendor has published the SSL certificate publicly, no need to add that manually.

TROUBLESHOOTING

• Field extractions not working / Search stops because of memory limit reached / Events are getting truncated/ Count mismatch on dashboards.
• Create a file 'limits.conf' in the following directory $SPLUNK_HOME/etc/TA-RiskIQ/local.
• Add the following 3 stanzas in the file:

      [kv]
      maxcols  = 1024
      limit    = 750
      maxchars = 50000
      max_extractor_time = 2000
      avg_extractor_time = 1000

      [mvexpand]
      max_mem_usage_mb =9000

      [lookup]
      max_memtable_bytes = 20000000

1. Restart Splunk
2. For the cluster environment, User needs to deploy limits.conf file on the Heavy forwarder, Indexer and Search Head.

SUPPORT

Contact - support@riskiq.com

Copyright 2016 - 2019 RiskIQ