icon/x Created with Sketch.

Splunk Cookie Policy

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website. Learn more (including how to update your settings) here.
Accept Cookie Policy

Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading ThrowAway
SHA256 checksum (throwaway_104.tgz) 1e069c87ad24b590d9a5dcff4f32486ab029663e093882cc9e8604556b28d723 SHA256 checksum (throwaway_103.tgz) 2983f26206cad96e75684699da4fa165770463236c2a49738741a2826a7b44eb SHA256 checksum (throwaway_102.tgz) e508699fcfffb6a045a6170615b25ec39eca7e26f1c2257c88fbbf154ca88d36
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

ThrowAway

Admins: Please read about Splunk Enterprise 8.0 and the Python 2.7 end-of-life changes and impact on apps and upgradeshere.
Overview
Details
This is a custom add-on that aims to give the Splunk Enterprise administrators control over the files on the local storage of Splunk Enterprise instances. The app can search for files/directories based on RegEx patterns and can remove them based on aging criteria.

Introduction

This is a custom add-on that aims to give the Splunk Enterprise administrators control over the files on the local storage of Splunk Enterprise instances. The app can search for files/directories based on RegEx patterns and can remove them based on aging criteria.

Deployment

The app is available in SPL and TAR bundles and these can be deployed quite easily through the GUI of single instance Splunk Enterprise installations. For distributed instances read below.

Non-clustered distributed environment

For non-clustered environments it is a matter of adding the un-archived bundle directory to the "deployment-apps" in the Deployment Server (DS) of the environment, followed by adding a server class mapping, either through the GUI of the DS or through editing the "serverclass.conf" file.

Clustered distributed environment

For environments with Indexer clustering the add-on should be first sent to the Cluster Master (CM) and then distributed via bundle replication to the Indexers/Peers. Every consequent configuration of the app should be performed via the CM. This will ensure that configuration changes are pushed to the peers gracefully and they are restarted one by one, without losing service availability.

Configuration scenarios

Example1 - Configure frozen bucket housekeeping

Housekeeping is performed on an index-level using the file/directory removal tool "TA-throw-away". You need to configure new stanza for every new index that is configured to keep data in frozen state for a certain time period. In case of distributed environment with indexer cluster you should perform this directly on the Cluster Master in order to deploy the configurations to all the cluster members.

  1. Login to your Cluster Master (for indexer cluster) or your Indexer (for non-clustered indexers) with your SSH client.
  2. Create/Edit the "inputs.conf" file in your "TA-throw-away" add-on:

sudo vi /opt/splunk/etc/master-apps/TA-throw-away/local/inputs.conf

  1. Use the template below to add a new housekeeping stanza in the "inputs.conf" file of the "TA-throw-away" for your index, e.g. for index "splunk_index":

[remove_files_directories://splunk_index]
index = _audit
interval = 86400
pattern = (d|r)b
\d{10,11}\d{10,11}.*
retention_policy = 1
retention_period = 32832000
timestamp_location = name
working_directory = /data1/frozen/splunk_index/frozendb/
disabled = 0_

  1. Save the file.
  2. For clustered indexers go back to the GUI of the Cluster Master, in the "Indexer Clustering" menu.
  3. Choose "Edit" → "Configuration Bundle Actions".
  4. Press the "Validate and Check Restart" button and wait for the process to finish.
    • Next, you need to press the "Push" button and wait the package to be deployed to each peer (might also perform automatic rolling restart).
    • After completing the restart, you can perform a search in the "_audit" index for events that point to directories being removed (if objects from the index frozen directory qualify).
  5. There's a known bug in Splunk, which prevents custom Python libraries to be loaded unless they are placed under "$SPLUNK_HOME$/etc/apps/". This will cause issues for "TA-throw-away" in clustered indexers. To alleviate the problem, you must access every indexer in the cluster through SSH client and create a symbolic link like below:
    sudo ln -s /opt/splunk/etc/slave-apps/TA-throw-away/ /opt/splunk/etc/apps/TA-throw-away

Example2 - Adding new cluster members

In order for the "TA-throw-away" add-on to work properly, you need to deploy it from the cluster master (which will happen automatically when you add a cluster member to the cluster). Then you also need to address a known limitation in Splunk, which is Splunk does not look for custom python code if the application that contains it is installed under anything different than "$SPLUNK_HOME$/etc/apps/". This is exactly what happens when the custom add-on is deployed from the Cluster Master - it ends up installed under "$SPLUNK_HOME$/etc/slave-apps". To work around this issue you should create a symbolic link under "$SPLUNK_HOME$/etc/apps" that points to the app in "$SPLUNK_HOME$/etc/slave-apps", e.g.:
sudo ln -s /opt/splunk/etc/slave-apps/TA-throw-away/ /opt/splunk/etc/apps/TA-throw-away

Configuration details

The snippet below shows complete documentation (please, read it carefully!) on the parameters used in the "inputs.conf" file:

#Replace all {vars} with your own and read the description carefully! Data will be deleted from disk!!!
#
#First is the stanza definition
#[remove_files_directories://{name_of_input}]
#
#Then configure the index, where events will be sent for storage. These events are holding the names of files/directories that were deleted.
#index = {name_of_index}
#
#The following parameter sets the frequency with which the input will be executed. Measured in seconds.
#interval = 86400
#
#Use ordinary RegEx pattern in the following parameter in order to select only matching files/directories for removal.
#E.g. (d|r)b_\d{10,11}_\d{10,11}_.* for Splunk index buckets.
#pattern = {RegEx}
#
#The next parameter will be used in order to remove files/directories based on their age. "1" means "enabled".
#retention_policy = 1
#
#The retention period is measured in seconds. It will be used in order to tell what is the maximum age of files/directories.
#Files/directories that are older than the maximum will be removed. E.g. if set to "86400" (these are seconds) this means that
#if the file/directory is older than 24 hours it will be qualified for removal.
#retention_period = 3600
#
#Timestamp location can be found in the name (in case of Splunk index buckets) or in the file/directory "last modified" attribute.
#Set to "name" if you would like to work with bucket names, or "last_modified" if you'd like to use the attribute of the file.
#timestamp_location = name
#
#Finally, provide the absolute path where you wish the input to look for files/directories. If you are using Splunk index buckets,
#make sure you provide the location to the frozendb directory for the particular index. Otherwise you might remove stuff which is currently being used…
#E.g. /opt/splunk/var/lib/splunk/defaultdb/frozendb/ for deleting frozen buckets from the "main" index. Path should always end with "/"!
#working_directory = {absolute_path}
#
#Use "1" if you wish to disable the input
#disabled = 0

Release Notes

Version 1.0.4
Oct. 14, 2020

Version 1.0.3
Oct. 13, 2020

Updated permissions in package to follow best-practices.

Version 1.0.2
Oct. 13, 2020

4
Installs
9
Downloads
Share Subscribe LOGIN TO DOWNLOAD

Subscribe Share

AppInspect Tooling

Splunk AppInspect evaluates Splunk apps against a set of Splunk-defined criteria to assess the validity and security of an app package and components.

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
© 2005-2020 Splunk Inc. All rights reserved.
Splunk®, Splunk>®, Listen to Your Data®, The Engine for Machine Data®, Hunk®, Splunk Cloud™, Splunk Light™, SPL™ and Splunk MINT™ are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners.