Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading Amazon Kinesis Modular Input
MD5 checksum (amazon-kinesis-modular-input_103.tgz) 575eec12cd53d351b0516b1f7d805555 MD5 checksum (amazon-kinesis-modular-input_102.tgz) dd209725b290c333f4d59e1fcbb15827 MD5 checksum (amazon-kinesis-modular-input_101.tgz) 1753e21dbdb0ebeb120aaf92bd223b5b MD5 checksum (amazon-kinesis-modular-input_10.tgz) f4d6a29bdba1eef811d45a6632fe8923 MD5 checksum (amazon-kinesis-modular-input_09.tgz) dabd3a8e6a5fdd81b24529ea88289efc MD5 checksum (amazon-kinesis-modular-input_08.tgz) 140264a4c0ed9367ef97f66a2053e38a MD5 checksum (amazon-kinesis-modular-input_07.tgz) 165745715e298c0d905ddb85c6c96e75 MD5 checksum (amazon-kinesis-modular-input_06.tgz) 1fca1cb4e759d50b0b6fa9f9f3d1d26f MD5 checksum (amazon-kinesis-modular-input_05.tgz) 9074367b6b4778d981c78c29c2500c2f
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

Amazon Kinesis Modular Input

Overview
Details
This is a Splunk Modular Input Add-On for indexing records from Amazon Kinesis

Splunk Amazon Kinesis Modular Input v1.0.3

Overview

This is a Splunk Modular Input Add-On for indexing records from Amazon Kinesis.

What is Kinesis ?

http://aws.amazon.com/kinesis/

Dependencies

  • Splunk 5.0+
  • Java Runtime 1.7+
  • Supported on Windows, Linux, MacOS, Solaris, FreeBSD, HP-UX, AIX

Setup

  • Optionally set your JAVA_HOME environment variable to the root directory of your JRE installation.If you don't set this , the input will look for a default installed java executable on the path.
  • Untar the release to your $SPLUNK_HOME/etc/apps directory
  • Restart Splunk

Configuration

As this is a Modular Input , you can then configure your Kinesis inputs via Manager->Data Inputs->Kinesis.

Logging

Any log entries/errors will get written to $SPLUNK_HOME/var/log/splunk/splunkd.log

JVM Heap Size

The default heap maximum is 64MB.
If you require a larger heap, then you can alter this in $SPLUNK_HOME/etc/apps/kinesis_ta/bin/kinesis.py on line 95

JVM System Properties

You can declare custom JVM System Properties when setting up new input stanzas.
Note : these JVM System Properties will apply to the entire JVM context and all stanzas you have setup

Customized Message Handling

The way in which the Modular Input processes the received Kinesis records is enitrely pluggable with custom implementations should you wish.

To do this you code an implementation of the com.splunk.modinput.kinesis.AbstractMessageHandler class and jar it up.

Ensure that the necessary jars are in the $SPLUNK_HOME/etc/apps/kinesis_ta/bin/lib directory.

If you don't need a custom handler then the default handler com.splunk.modinput.kinesis.DefaultMessageHandler will be used.

Code examples are on GitHub : https://github.com/damiendallimore/SplunkModularInputsJavaFramework/tree/master/kinesis/src/com/splunk/modinput/kinesis

Troubleshooting

  • JAVA_HOME environment variable is set or "java" is on the PATH for the user's environment you are running Splunk as
  • You are using Splunk 5+
  • You are using a 1.7+ Java Runtime
  • You are running on a supported operating system
  • Look for any errors in $SPLUNK_HOME/var/log/splunk/splunkd.log
  • Run this command as the same user that you are running Splunk as and observe console output : "$SPLUNK_HOME/bin/splunk cmd python ../etc/apps/kinesis_ta/bin/kinesis.py --scheme"

Contact

This project was initiated by Damien Dallimore , damien@baboonbones.com

Release Notes

Version 1.0.3
April 21, 2016

Added JSON Object parsing for Cloudwatch to the GZIP handler

Version 1.0.2
April 16, 2016

tweaks to gzip handler

Version 1.0.1
April 13, 2016

Pushed default charset decoding out of the main message processing flow and into custom handling , so custom handlers that you implement should in theory be able to process any binary or text payload.

Version 1.0
April 12, 2016

Can now pass the raw payload bytes to your custom message handler ie: if you want to decode binary data
Added a custom GZIP decoder , com.splunk.modinput.kinesis.GZIPDataRecordDecoderHandler

Version 0.9
Nov. 10, 2015

Tweaked the HEC transport.

Added a new custom handler that allows you to declare the fieldnames in the JSON that hold the time and host values of the event.

message_handler_impl = com.splunk.modinput.kinesis.JSONBodyWithFieldExtraction
message_handler_params = timefield=foo,hostfield=goo

Version 0.8
Sept. 22, 2015

Added support to optional output to Splunk via a HEC (HTTP Event Collector) endpoint

Version 0.7
Feb. 11, 2015

Enabled TLS1.2 support by default.
Made the core Modular Input Framework compatible with latest Splunk Java SDK
Please use a Java Runtime version 7+
If you need to use SSLv3 , you can turn this on in bin/kinesis.py
SECURE_TRANSPORT = "tls"
#SECURE_TRANSPORT = "ssl"

Version 0.6
Sept. 12, 2014

Added a custom message handler that just dumps the JSON body.

message_handler_impl = com.splunk.modinput.kinesis.JSONOnlyMessageHandler

Version 0.5
Aug. 30, 2014

Initial Beta release

22
Installs
705
Downloads
Share Subscribe LOGIN TO DOWNLOAD

Subscribe Share

Splunk Certification Program

Splunk's App Certification program uses a specific set of criteria to evaluate the level of quality, usability and security your app offers to its users. In addition, we evaluate the documentation and support you offer to your app's users.

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 50GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
© 2005-2017 Splunk Inc. All rights reserved.
Splunk®, Splunk>®, Listen to Your Data®, The Engine for Machine Data®, Hunk®, Splunk Cloud™, Splunk Light™, SPL™ and Splunk MINT™ are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners.