Splunk JMS Modular Input v1.5.1
by Damien T. Dallimore
This is a Splunk modular input add-on for polling message queues and topics via the JMS interface.
Why JMS ?
JMS is simply a messaging API and is a convenient means by which to write 1 modular input that can talk to several different underlying messaging providers.
The modular input code is generic because it is programmed to the JMS interface.
You can then supply messaging provider specific jar files at runtime.
More details on JMS at Wikipedia,
As this is a modular input , you can then configure your JMS inputs via Manager->DataInputs
JNDI vs Local mode
For the most part you will setup your JMS connectivity using JNDI to obtain the remote JMS objects.
However, you can bypass JNDI if you wish and use local instantiation.
To do this you must code an implementation of the com.splunk.modinput.jms.LocalJMSResourceFactory interface.
You can then bundle the classes in a jar file and place them in $SPLUNK_HOME/etc/apps/jms_ta/bin/lib
The configuration screen in Splunk Manager for creating a new JMS input allows you to choose local or jndi as the instantiation mode.
So choose local , and then you can specify the name of implementation class, as well as any declarative parameters you want to pass in.
Any log entries/errors will get written to $SPLUNK_HOME/var/log/splunk/splunkd.log
Third party jars
If you require specific JMS provider or JNDI Context implementation jars, then you can simply copy these to $SPLUNK_HOME/etc/apps/jms_ta/bin/lib
They will be automatically picked up upon restart
JVM Heap Size
The default heap maximum is 64MB.
If you require a larger heap, then you can alter this in $SPLUNK_HOME/etc/apps/jms_ta/bin/jms.py on line 95
JVM System Properties
You can declare custom JVM System Properties when setting up new input stanzas.
Note : these JVM System Properties will apply to the entire JVM context and all stanzas you have setup
Added a new message handler that just dumps the message body :
Minor HEC data handling tweaks
Added support to optional output to Splunk via a HEC (HTTP Event Collector) endpoint
Enabled TLS1.2 support by default.
Made the core Modular Input Framework compatible with latest Splunk Java SDK
Please use a Java Runtime version 7+
If you need to use SSLv3 , you can turn this on in bin/jms.py
SECURE_TRANSPORT = "tls"
#SECURE_TRANSPORT = "ssl"
Changed the point in the code where client ID is set for durable topic subscriptions
Added a LocalConnectionFactory for ActiveMQ
Added the ability to declare custom JVM System Properties in your stanzas
Minor cosmetic fix
Added system property javax.net.ssl.trustStorePassword to LocalMQConnectionFactory
Added a custom local JMS resource factory for Websphere MQ users that allows you to create a MQConnectionFactory class instance directly vs looking it up via JNDI.
The implementation class name you can declare in you stanza is : com.splunk.modinput.jms.custom.factory.LocalMQConnectionFactory
It depends on the MQ JMS jar files being in SPLUNK_HOME/etc/apps/jms_ta/bin/lib
Parameter values supported are (showing example values) :
You declare these in the stanza also in a key=value string , comma delimited :
Changed suffix to .spl
Removed field validation constraints when selecting local mode
Updated configuration screen , hopefully it is simpler and more intuitive
More verbose error logging ++ Removed usage of hostname for REST callbacks and replaced with localhost to alleviate issues where admins haven't setup local DNS correctly
Added support for very large message payloads, basically a refactoring of the XML streaming output so that SplunkD can correctly process large message payloads
Refactored the core engine so that now you can declare a pluggable JMS Message handler.For the most part you'll probably just rely on the default message handler that is part of the mod input. But given that a message producer can basically put anything in the message payload, there may be scenarios where you need some custom handling for the raw message payload ,so this new features provides this flexibility.
Patched a hole whereby the "disabled" property is not passed to the Mod Input by SplunkD when it is first created
Added more detailed error logging (written to $SPLUNK_HOME/var/log/splunk/splunkd.log) for troubleshooting
Fixed a possible(in theory) race condition scenario at startup ++ Added support for Queue Browsing , user can choose browsing mode(dump all messages or output summary stats about the queue) and the browsing frequency.
Some core framework tweaks++Added config option so users can specify whether or not to strip newline characters from the message body
Added runtime support for solaris, aix, freebsd, hp-ux***Added support to prefix your destination name with a server identifier , which now makes it possible to setup jms stanzas that have the same destination name but on different servers
Added REST API polling calls to the mod input so that it can poll SplunkD for the enabled/disabled status of each queue/topic stanza and stop message polling threads accordingly or terminate the entire mod input process if all queue/topic stanzas have been disabled.Using the Splunk Java SDK v1.0 for the REST API call, hence splunk.jar is now included in this release.
Added better handling of Binary message types
Added support for providing user/pass for the topic/queue
Cosmetic fix, added screenshot.
Splunk's App Certification program uses a specific set of criteria to evaluate the level of quality, usability and security your app offers to its users. In addition, we evaluate the documentation and support you offer to your app's users.
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 50GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.