Author: Joe Welsh - Splunk
Version/Date: 1.13 July 14, 2020
Supported product(s):
This add-on supports logs containing HL7 v2.X data.
Source type(s): This add-on will process data that is source-typed
as "hl7_v2".
Input requirements: N/A
To be used with HL7 Dashboard Examples App for Splunk ( )
Configuration
To use this add-on, manually configure the data input with the following input.conf
properties and the add-on will optimize the data automatically.
sourcetype = hl7_v2
These properties will also be utilized by the underlying searches and eventtypes .
The value from this Add-on is derived by it's ability to:
If the logs come from Cloverleaf's Interface engine there are additional extractions for thread_id, client ip, and client port.
Notes:
*The datetime format can vary for each HL7 message. In lieu of using TIME_FORMAT inside props.conf
a custom datetime.xml was created inside this app. The extracted groups correspond to those specified
in the extracted attribute for define element. The groups (year, month, day, etc.) are specified in
Splunk's default datetime.xml in /etc/system/datetime.xml.
**Repeating Segments will be placed inside multi-value fields.
i.e. If there are four NTE segments in the message there will be one multi-value NTE field with four values.
The subsequent HL7 fields will only be extracted for the first segment in repeating segments.
The reference files used for this Add-on can be found in the cloverleaf_logs directory.
The samples directory contains files used to generate sample messages. If you install Splunk's Eventgen App v7.2+ [https://splunkbase.splunk.com/app/1924/#/overview] these samples will be replayed and put in your main index.
You'll also need to modify splunkUser and splunkPass in eventgen.conf to match credentials on your system.
HL7_Segment_Template transformation uses MV_ADD = true to aggregate repeating segments. However, you ultimately need to construct a mv field using the makemv SPL command to split them out for use. However, I was unable to split it using the newline character as a delimiter, needed to add an anchor to the end of each line first.
So, take this sample message with repeating DG1 segments:
MSH|^~\&|EPIC|MGH||MGH|201503241909|OHEDSCRIBE|ADT^A08|725467|T|2.3|||||||||
EVN|A08|20150324190937||REG_UPDATE|^EMERGENCY^SCRIBE^OH^^^^^OHSA^^^^^MGH||
PID|1||3100068701^^^EPI^MR||MGHTEST^EDSCRIBE||19600324|M|||^^^^^US^P|||||||6100215419|999-99-9999|||||||||||N||
PD1|||MARION GENERAL HOSPITAL^^1016100|||||||||||||||
PV1|1|E|ED^ED1^01^MGH^R^^^^^^DEPID||||15802^BOEHMER^DAVID^C^^^^^STARPROV^^^^STARPROV|||Emergency||||||||EMERGENCY|6100215419|SELF||||||||||||||||||||||^^^MGH^^^^^^^||20150324190300||||||27150830003
PV2||GENERAL||||||||||||||||||||N|||||||||||||||||||||||||||
ZPV|||||||||||||20150324190832||||||||
AL1|1|Drug Class|45202^NO KNOWN ALLERGIES^||||||
DG1|1||52^Headache^HRV|Headache||
DG1|2||53^Neck^NCK|Neck||
DG1|3||54^Ear^EAR|Outer Ear||
GT1|1|100046662|MGHTEST^EDSCRIBE^^||^^^^^US|||19600324|M|P/F|SLF|999-99-9999|||||||||||||||||||||||||||||||||||||
ZMP|1|||||N||||^^^^|||^^^^|||||^^^^||||^^^^||Not eligible for Medicare||||^^^^|||||^^^^|0|||^^^^|||||||||||||||||||||||||||||
T_50005_1T_50005_1{CONNID 0} {IPVERSION 4} {CLIENTIP 172.17.154.197} {CLIENTPORT 63143}
HL7_Segment_Template transform, with MV_ADD = true, collected ALL the DG1 segments and put the aggregated values as the single value for DG1:
1||52^Headache^HRV|Headache||
2||53^Neck^NCK|Neck||
3||54^Ear^EAR|Outer Ear||
Then I ran this SPL: sourcetype="hl7_v2" DG1=*| rex mode=sed field=DG1 "s/$/:/g"
which tacks on a ":" delimiter to end of the line and produces:
1||52^Headache^HRV|Headache||:
2||53^Neck^NCK|Neck||:
3||54^Ear^EAR|Outer Ear||:
This SPL then combines the three segments into single line, using end of line as anchor:
sourcetype="hl7_v2" DG1=* | rex mode=sed field=DG1 "s/$/:/g" | makemv delim="$" DG1
1||52^Headache^HRV|Headache||:2||53^Neck^NCK|Neck||:3||54^Ear^EAR|Outer Ear||:
Then I employ the split function multiple times within an eval to split DG1, and use mvindex to extract the particular iterated DG1 segment:
sourcetype="hl7_v2" DG1=* | rex mode=sed field=DG1 "s/$/:/g" | makemv delim="$" DG1| eval s1=mvindex(split('DG1', ":"),0), s2=mvindex(split('DG1', ":"),1), s3=mvindex(split('DG1', ":"),2)
If you're not seeing extracted field in your searches try adding "| fields *" to the end of your search.
-Now supports Splunk 8.x
-Will be Splunk Cloud Certified
-Removed pre-built panels and dashboards. Will create a new, HL7 Analytics App to be used with Add-on
-Changed permissions required for Splunk Cloud Vetting
find . -type d -exec chmod 0755 {} \;
find . -type f -exec chmod 0644 {} \;
-Updated macros to work properly in Splunk 7.x
-Updated eventgen.conf to ensure it works with latest Eventgen in Splunkbase: https://splunkbase.splunk.com/app/1924/
-Ensured all three pre-built panels work as a result of all changes
-Updated Message Type Lookup ( hl7_v2_message_types.csv). Please use Message_Type_Description in lieu of previous Message_Type in your searches.
- Updated files in samples to replay messages with missing patient names and identifiers.
- Put SEDCMD back to replace carriage returns with line feeds.
- Removed indexes.conf
- Added instructions to README.md for extracting individual segments when they are repeating.
- Modified files in samples for eventgen to populate the default/main index instead of previously used hl7 index.
- Modified LINE_BREAKER in props.conf to properly parse HL7 messages that contain "MSH" in the actual message, not just the three first characters.
- Commented out SEDCMD in props.conf for replacing carriage return with line feed.
- Added new sample file in cloverleaf_logs
=== Splunk TA HL7 (Add-On) ===
Author: Joe Welsh - Splunk
Version/Date: 1.05 August 25, 2016
*Note: This app was previously Add-on for HL7
Supported product(s):
This add-on supports logs containing HL7 v2.X data.
Source type(s): This add-on will process data that is source-typed
as "hl7_v2".
Input requirements: N/A
Splunk AppInspect evaluates Splunk apps against a set of Splunk-defined criteria to assess the validity and security of an app package and components.
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.