Overview

Details

This is a Splunk Modular Input for polling data from REST APIs and indexing the responses

Splunk REST API Modular Input v1.5.7

Overview

This is a Splunk modular input add-on for polling REST APIs.

Activation Key

You require an activation key to use this App. Visit http://www.baboonbones.com/#activation to obtain a non-expiring key

Features

Perform HTTP(s) GET/POST/PUT/HEAD requests to REST endpoints and output the responses to Splunk
Multiple authentication mechanisms
Add custom HTTP(s) Header properties
Add custom URL arguments
HTTP(s) Streaming Requests
HTTP(s) Proxy support , supports HTTP CONNECT Verb
Response regex patterns to filter out responses
Configurable polling interval
Configurable timeouts
Configurable indexing of error codes
Persist and retrieve cookies

Authentication

The following authentication mechanisms are supported:

None
HTTP Basic
HTTP Digest
OAuth1
OAuth2 (with auto refresh of the access token)
Custom

Custom Authentication Handlers

You can provide your own custom Authentication Handler. This is a Python class that you should add to the rest_ta/bin/authhandlers.py module.

http://docs.python-requests.org/en/latest/user/advanced/#custom-authentication

You can then declare this class name and any parameters in the REST Input setup page.

Custom Response Handlers

You can provide your own custom Response Handler. This is a Python class that you should add to the rest_ta/bin/responsehandlers.py module.

You can then declare this class name and any parameters in the REST Input setup page.

Token substitution in Endpoint URL

There is support for dynamic token substitution in the endpoint URL

ie : /someurl/foo/$sometoken$/goo

$sometoken$ will get substituted with the output of the 'sometoken' function in bin/tokens.py

So you can add you own tokens simply by adding a function to bin/tokens.py

Currenty there is 1 token implemented , $datetoday$ , which will resolve to today's date in format "2014-02-18"

Token replacement functions in the URL can also return a list of values, that will cause
multiple URL's to be formed and the requests for these URL's will be executed in parallel in multiple threads.

Dependencies

Splunk 5.0+
Supported on Windows, Linux, MacOS, Solaris, FreeBSD, HP-UX, AIX

Setup

Untar the release to your $SPLUNK_HOME/etc/apps directory
Restart Splunk
If you are using a Splunk UI Browse to Settings -- Data Inputs -- REST to add a new Input stanza via the UI
If you are not using a Splunk UI (ie: you are running on a Universal Forwarder) , you need to add a stanza to inputs.conf directly as per the specification in README/inputs.conf.spec. The inputs.conf file should be placed in a local directory under an App or User context.

Logging

Any log entries/errors will get written to $SPLUNK_HOME/var/log/splunk/splunkd.log

These are also searchable in Splunk : index=_internal error rest.py

Troubleshooting

You are using Splunk 5+
Look for any errors in $SPLUNK_HOME/var/log/splunk/splunkd.log
Any firewalls blocking outgoing HTTP calls
Is your REST URL, headers, url arguments correct
Is you authentication setup correctly

Contact

www.baboonbones.com

Release Notes

Version 1.5.7

Aug. 13, 2019

added client certificate config options

Version 1.5.6

April 23, 2019

updated docs

Version 1.5.5

April 19, 2019

added trial key functionality

Version 1.5.4

March 6, 2019

added a triggers stanza to app.conf to prevent reloading after saving state back to inputs.conf

Version 1.5.3

June 4, 2018

Patched a bug to callbacks to Splunk for persisting state that required the activation key in the payload

Version 1.5.2

June 3, 2018

minor manager xml ui tweak for 7.1

Version 1.5.1

May 30, 2018

Corrected a build bug with responsehandlers

Version 1.5

May 27, 2018

Added an activation key requirement , visit http://www.baboonbones.com/#activation to obtain a free,non-expiring key
Added support for HEAD requests
Docs updated
Splunk 7.1 compatible

Version 1.4

Sept. 2, 2015

Delimiter fix

Version 1.3.9

July 15, 2015

Can now declare a CRON pattern for your polling interval.
Multiple requests spawned by tokenization can be declared to run in parallel or sequentially.
Multiple sequential requests can optionally have a stagger time enforced between each request.

Version 1.3.8

July 12, 2015

Minor code fix for a logging statement error

Version 1.3.7

July 3, 2015

Added support for token replacement functions in the URL to be able to return a list
of values, that will cause multiple URL's to be formed and the requests for these
URL's will be executed in parallel in multiple threads. See tokens.py

Version 1.3.6

Jan. 27, 2015

Added a custom response handler for rolling out generic JSON arrays
Refactored key=value delimited string handling to only split on the first "=" delimiter

Version 1.3.5

Aug. 20, 2014

Ensure that token substitution in the endpoint URL is dynamically applied for each
HTTP request

Version 1.3.4

Feb. 18, 2014

Added support for dynamic token substitution in the endpoint URL

ie : /someurl/foo/$sometoken$/goo

$sometoken$ will get substituted with the output of the 'sometoken' function
in bin/tokens.py

Currently have just shipped with 1 example token $datetoday$ which will dynamically resolve to today's date in format 2014-02-18

Version 1.3.3

Feb. 14, 2014

Added support for sending and persisting cookies

Version 1.3.2

Oct. 30, 2013

Changed the logic for persistence of state back to inputs.conf to occur directly after polling/event indexing has completed rather than waiting for the polling loop frequency sleep period to exit. This potentially deals with situations where you might terminate Splunk before the REST Mod Input has persisted state changes back to inputs.conf because it was in a sleep loop during shutdown.

Version 1.3.1

Oct. 24, 2013

Cosmetic fix for 1.3 release

Version 1.3

Oct. 24, 2013

Added a new feature that will automatically persist updates to URL Arguments , HTTP Header Propertys or HTTP Request Body content back to your inputs.conf stanza. Such a scenario might occur if you are using a custom response handler to dynamically calculate URL Arguments , such as a timestamp or event paging cursor, and you want this latest state to be persisted back into your configuration so that if you need to restart the REST input , it's configuration is in the latest polled state and can resume polling from where it left off.

Version 1.2

Oct. 16, 2013

Upgraded underlying python requests library to version 2.0 , primarily to support the HTTP CONNECT verb

Version 1.1

Aug. 14, 2013

Added support for user defined delimiter for multiple "key=value" fields .
Added hooks in responsehandlers.py for custom handling of responses, use cases such as URL arguments/HTTP header properties that might require a dynamic value per request , HTTP REL Header link following , dynamically changing the endpoint URL.

Version 1.0.6beta

July 10, 2013

Strip newlines from default output ++ add unbroken attribute to XML output stream so that Splunk props /transforms can be applied

Version 1.0.5beta

July 8, 2013

Added more robust exception handling

Version 1.0.4beta

June 25, 2013

Fixed minor script bug when printing http errors

Version 1.0.3beta

June 25, 2013

Added support for POST and PUT HTTP Methods for getting data. Not RESTful per say but a useful out for API's that are "REST like"

Version 1.0.2beta

June 20, 2013

Renamed the manager xml file to avoid naming clashes

Version 1.0.1beta

June 17, 2013

Fixed some spelling typos

Version 1.0beta

June 10, 2013

First release

2,528

Installs

28,265

Downloads

Share Subscribe LOGIN TO DOWNLOAD

Version

Built by

BaboonBones Ltd

Contributors

BaboonBones Ltd

Support

Developer Supported

Contact Developer Questions on Splunk Answers Flag as inappropriate

App Contents: Inputs