The Splunk DFS Manager app offers the easiest way of deploying Spark automatically to run DFS searches irrespective of your search topology. Therefore, Splunk only provides support for a Spark cluster that is deployed using the Splunk DFS Manager app. If you install your Spark cluster manually, Splunk isn't responsible for support or maintenance of the compute cluster.
For information on installing and configuring the app for Splunk DFS Manager, see Splunk Docs.
You can use the app irrespective of your deployment scenario and install Spark on a standalone search head, standalone indexer, search head cluster, or an indexer cluster. The Splunk DFS Manager app must be installed on the search head and all the search peers that are a part of the DFS deployment. In a search head cluster, the app must be installed on all the search heads. In an indexer cluster, the app must be deployed to all the members of the indexer cluster using the cluster master node. Ensure that the app binaries are available on the required DFS search heads and the search peers that you want to use as DFS workers.
Supported topologies and installation paths:
- Search head clusters: $SPLUNK_HOME/etc/shcluster
- Indexer clusters: $SPLUNK_HOME/etc/master-apps and then bundled push to: $SPLUNK_HOME/etc/apps
- Standalone search head: $SPLUNK_HOME/etc/apps
- Indexers: $SPLUNK_HOME/etc/apps
Following are some of the common issues that you may see while using the Splunk DFS Manager app:
- Splunk DFS Manager UI does not load
- DFS master or DFS worker does not start
- Removing or adding DFSworker fails
For more information on troubleshooting error messages, see Splunk DFS Manager Error Messages.
Contact Splunk Services if you run into issues trying to configure your Spark cluster using the app for Splunk DFS Manager.
Please post a question on the Splunk DFS Manager app at Splunk Answers.
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.