Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading REST API Modular Input
MD5 checksum (rest-api-modular-input_14.tgz) 9f474c4604cd85782f572262f9036b0a MD5 checksum (rest-api-modular-input_139.tgz) 5e5c8a52611bcafee4ec93140065b3cc MD5 checksum (rest-api-modular-input_138.tgz) fa40c3aa65e0554ad89c488a1104a401 MD5 checksum (rest-api-modular-input_137.tgz) 8514ef03719cd7d887d02b2c4ef7a581 MD5 checksum (rest-api-modular-input_136.tgz) 7d1ef9e6e942bf71a7723a789509616e MD5 checksum (rest-api-modular-input_135.tgz) 55e2ebb1aa8812c70028cde9d63ab739 MD5 checksum (rest-api-modular-input_134.tgz) 061896a146168d1c0d86e1162a2d18ee MD5 checksum (rest-api-modular-input_133.tgz) 90969fc4a96792722f5b13a974606b56 MD5 checksum (rest-api-modular-input_132.tgz) 6c8023b96123822169514b694cd0c928 MD5 checksum (rest-api-modular-input_131.tgz) ccf4d26d3f092daf7c30201e6234c9b5 MD5 checksum (rest-api-modular-input_13.tgz) 94de78b41fe4a419308ba04f1627c771 MD5 checksum (rest-api-modular-input_12.tgz) 6ab92f6eaf6295cf08c461296f6c4afa MD5 checksum (rest-api-modular-input_11.tgz) 31c396d0cd09ee50092537aae8cea180 MD5 checksum (rest-api-modular-input_106beta.tgz) f696c21bf4c12bb5213955c304e042d9 MD5 checksum (rest-api-modular-input_105beta.tgz) f76bed6b65cce9b1a0c6884eb72e3c8e MD5 checksum (rest-api-modular-input_104beta.tgz) 1329f452974c56ed753e85a1f0e9b069 MD5 checksum (rest-api-modular-input_103beta.tgz) 699541a135e4a0fd274b955623ab55dd MD5 checksum (rest-api-modular-input_102beta.tgz) 1b4a92fb3afb7c311513e486526614a8 MD5 checksum (rest-api-modular-input_101beta.tgz) 365dd9dc023456827fec0fb3dff11e2b MD5 checksum (rest-api-modular-input_10beta.tgz) c6431c8f647f22fc8e4988a2d0bf71ee
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

REST API Modular Input

Overview
Details
This is a Splunk Modular Input for polling data from REST APIs and indexing the responses

Splunk REST API Modular Input v1.4

by Damien Dallimore , damien@baboonbones.com

Overview

This is a Splunk Modular Input for polling REST APIs and indexing the responses.

Features

  • Perform HTTP(s) GET requests to REST endpoints and output the responses to Splunk
  • Optional POST and PUT HTTP Methods
  • Multiple authentication mechanisms
  • Add custom HTTP(s) Header properties
  • Add custom URL arguments
  • HTTP(s) Streaming Requests
  • HTTP(s) Proxy support
  • Response regex patterns to filter out responses
  • Configurable polling interval
  • Configurable timeouts
  • Configurable indexing of error codes
  • Custom Response Handling plugin architecture
  • Persist and retrieve cookies

Authentication

The following authentication mechanisms are supported:

  • None
  • HTTP Basic
  • HTTP Digest
  • OAuth1
  • OAuth2 (with auto refresh of the access token)
  • Custom

Custom Authentication Handlers

You can provide your own custom Authentication Handler. This is a Python class that you should add to the
rest_ta/bin/authhandlers.py module.

http://docs.python-requests.org/en/latest/user/advanced/#custom-authentication

You can then declare this class name and any parameters in the REST Input setup page.

Custom Response Handlers

You can provide your own custom Response Handler. This is a Python class that you should add to the
rest_ta/bin/responsehandlers.py module.

You can then declare this class name and any parameters in the REST Input setup page.

Such use cases for implementing a custom response handler might include

  • dynamic setting of URL arguments/HTTP header propertys ie: a stream cursor
  • extracting the next link to follow from the response content or HTTTP REL header
  • custom pre-processing of the raw HTTP response before indexing in Splunk
  • dynamically changing the REST endpoint URL

Token substitution in Endpoint URL

There is support for dynamic token substitution in the endpoint URL

ie : /someurl/foo/$sometoken$/goo

$sometoken$ will get substituted with the output of the 'sometoken' function in bin/tokens.py

So you can add you own tokens simply by adding a function to bin/tokens.py

Currenty there is 1 token implemented , $datetoday$ , which will resolve to today's date in format "2014-02-18"

Token replacement functions in the URL can also return a list of values, that will cause
multiple URL's to be formed and the requests for these URL's will be executed in parallel in multiple threads.

Dependencies

  • Splunk 5.0+
  • Supported on Windows, Linux, MacOS, Solaris, FreeBSD, HP-UX, AIX

Setup

  • Untar the release to your $SPLUNK_HOME/etc/apps directory
  • Restart Splunk
  • Browse to Manager -> Data Inputs -> REST and setup your inputs

Logging

Any modular input log errors will get written to $SPLUNK_HOME/var/log/splunk/splunkd.log

Troubleshooting

  • You are using Splunk 5+
  • Look for any errors in $SPLUNK_HOME/var/log/splunk/splunkd.log
  • Any firewalls blocking outgoing HTTP calls
  • Are your REST URL, headers, url arguments correct
  • Is your authentication setup correctly

Release Notes

Version 1.4
Sept. 2, 2015

Delimiter fix

Version 1.3.9
July 15, 2015

Can now declare a CRON pattern for your polling interval.
Multiple requests spawned by tokenization can be declared to run in parallel or sequentially.
Multiple sequential requests can optionally have a stagger time enforced between each request.

Version 1.3.8
July 12, 2015

Minor code fix for a logging statement error

Version 1.3.7
July 3, 2015

Added support for token replacement functions in the URL to be able to return a list
of values, that will cause multiple URL's to be formed and the requests for these
URL's will be executed in parallel in multiple threads. See tokens.py

Version 1.3.6
Jan. 27, 2015

Added a custom response handler for rolling out generic JSON arrays
Refactored key=value delimited string handling to only split on the first "=" delimiter

Version 1.3.5
Aug. 20, 2014

Ensure that token substitution in the endpoint URL is dynamically applied for each
HTTP request

Version 1.3.4
Feb. 18, 2014

Added support for dynamic token substitution in the endpoint URL

ie : /someurl/foo/$sometoken$/goo

$sometoken$ will get substituted with the output of the 'sometoken' function
in bin/tokens.py

Currently have just shipped with 1 example token $datetoday$ which will dynamically resolve to today's date in format 2014-02-18

Version 1.3.3
Feb. 14, 2014

Added support for sending and persisting cookies

Version 1.3.2
Oct. 30, 2013

Changed the logic for persistence of state back to inputs.conf to occur directly after polling/event indexing has completed rather than waiting for the polling loop frequency sleep period to exit. This potentially deals with situations where you might terminate Splunk before the REST Mod Input has persisted state changes back to inputs.conf because it was in a sleep loop during shutdown.

Version 1.3.1
Oct. 24, 2013

Cosmetic fix for 1.3 release

Version 1.3
Oct. 24, 2013

Added a new feature that will automatically persist updates to URL Arguments , HTTP Header Propertys or HTTP Request Body content back to your inputs.conf stanza. Such a scenario might occur if you are using a custom response handler to dynamically calculate URL Arguments , such as a timestamp or event paging cursor, and you want this latest state to be persisted back into your configuration so that if you need to restart the REST input , it's configuration is in the latest polled state and can resume polling from where it left off.

Version 1.2
Oct. 16, 2013

Upgraded underlying python requests library to version 2.0 , primarily to support the HTTP CONNECT verb

Version 1.1
Aug. 14, 2013

Added support for user defined delimiter for multiple "key=value" fields .
Added hooks in responsehandlers.py for custom handling of responses, use cases such as URL arguments/HTTP header properties that might require a dynamic value per request , HTTP REL Header link following , dynamically changing the endpoint URL.

Version 1.0.6beta
July 10, 2013

Strip newlines from default output ++ add unbroken attribute to XML output stream so that Splunk props /transforms can be applied

Version 1.0.5beta
July 8, 2013

Added more robust exception handling

Version 1.0.4beta
June 25, 2013

Fixed minor script bug when printing http errors

Version 1.0.3beta
June 25, 2013

Added support for POST and PUT HTTP Methods for getting data. Not RESTful per say but a useful out for API's that are "REST like"

Version 1.0.2beta
June 20, 2013

Renamed the manager xml file to avoid naming clashes

Version 1.0.1beta
June 17, 2013

Fixed some spelling typos

Version 1.0beta
June 10, 2013

First release

1,258
Installs
14,519
Downloads
Share Subscribe LOGIN TO DOWNLOAD

Subscribe Share

Splunk Certification Program

Splunk's App Certification program uses a specific set of criteria to evaluate the level of quality, usability and security your app offers to its users. In addition, we evaluate the documentation and support you offer to your app's users.

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 50GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
© 2005-2017 Splunk Inc. All rights reserved.
Splunk®, Splunk>®, Listen to Your Data®, The Engine for Machine Data®, Hunk®, Splunk Cloud™, Splunk Light™, SPL™ and Splunk MINT™ are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners.