icon/x Created with Sketch.

Splunk Cookie Policy

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website. Learn more (including how to update your settings) here.
Accept Cookie Policy

Accept License Agreements

Thank You

Downloading Splunk DFS Manager
SHA256 checksum (splunk-dfs-manager_120.tgz) 43d319744c1a33eb943164e8c62ca546b7ee2b820e815a591248074be677f1b9
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

Splunk DFS Manager

Splunk Built
Admins: Please read about Splunk Enterprise 8.0 and the Python 2.7 end-of-life changes and impact on apps and upgradeshere.
The Splunk DFS Manager app enables you to run Data Fabric Search (DFS). The app automatically bundles, installs, and configures Apache Spark that is required for DFS. Additionally, the app helps you to continuously and seamlessly manage, configure, and monitor your Spark cluster and customize the resource allocation within your DFS deployment from any search head through a scalable and adaptive user interface to run a DFS search.

Splunk DFS Manager installation and configuration:
1) Decide how many workers are required.
2) Provision nodes to run as workers.
3) Add the nodes as distributed search peers to the search head.
4) Install the app on the search head and search peers.
- For indexers in an indexer cluster: Install the app using a cluster master.
- For search heads in a search head cluster:Install the app using a search head cluster deployer.
5) DFSmaster automatically starts on the search head (in standalone mode) and on the search head captain in a search head cluster.
6) Access the Splunk DFS manager UI on the search head that runs as the DFS master.
7) Add the provisioned search peers as the DFS workers.
8) Run a DFS search.

Splunk DFS Manager capabilities:
- High availability that allows an alternate search head captain to restart the Spark master, in case the search head captain fails.
- Customize resource usage by leveraging workload management (WLM) to allocate CPU and memory.
- Add DFS workers on all or selected search peers.
- Remove a DFS worker.
- Restrict adding DFS workers to a particular site in a multi-site environment.
- Monitor the health and resource usage of the Spark cluster.
- Enable Spark security.

The Splunk DFS Manager app offers the easiest way of deploying Spark automatically to run DFS searches irrespective of your search topology. Therefore, Splunk only provides support for a Spark cluster that is deployed using the Splunk DFS Manager app. If you install your Spark cluster manually, Splunk isn't responsible for support or maintenance of the compute cluster.

For information on installing and configuring the app for Splunk DFS Manager, see Splunk Docs.

Application Details

  • Supported Splunk Enterprise: 8.0
  • Supported Spark version: 2.3.3

Installation Instructions

You can use the app irrespective of your deployment scenario and install Spark on a standalone search head, standalone indexer, search head cluster, or an indexer cluster. The Splunk DFS Manager app must be installed on the search head and all the search peers that are a part of the DFS deployment. In a search head cluster, the app must be installed on all the search heads. In an indexer cluster, the app must be deployed to all the members of the indexer cluster using the cluster master node. Ensure that the app binaries are available on the required DFS search heads and the search peers that you want to use as DFS workers.

Supported topologies and installation paths:
- Search head clusters: $SPLUNK_HOME/etc/shcluster
- Indexer clusters: $SPLUNK_HOME/etc/master-apps and then bundled push to: $SPLUNK_HOME/etc/apps
- Standalone search head: $SPLUNK_HOME/etc/apps
- Indexers: $SPLUNK_HOME/etc/apps


Following are some of the common issues that you may see while using the Splunk DFS Manager app:
- Splunk DFS Manager UI does not load
- DFS master or DFS worker does not start
- Removing or adding DFSworker fails

For more information on troubleshooting error messages, see Splunk DFS Manager Error Messages.

Getting Help:
Contact Splunk Services if you run into issues trying to configure your Spark cluster using the app for Splunk DFS Manager.
Please post a question on the Splunk DFS Manager app at Splunk Answers.

Release Notes

Version 1.2.0
Feb. 19, 2020



Subscribe Share

AppInspect Tooling

Splunk AppInspect evaluates Splunk apps against a set of Splunk-defined criteria to assess the validity and security of an app package and components.

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
© 2005-2020 Splunk Inc. All rights reserved.
Splunk®, Splunk>®, Listen to Your Data®, The Engine for Machine Data®, Hunk®, Splunk Cloud™, Splunk Light™, SPL™ and Splunk MINT™ are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners.