HL7 Add-On for Splunk
Overview
Details
The value from this Add-on is derived by it's ability to:
1. Parse every HL7 message from log entry.*
2. Extract every segment within any HL7 v2.x message into it's own Splunk Field.**
3. Extract every field within every segment in the message.**
4. Provide examples on how to extract values from HL7 subfields.
i.e. PID-5 contains family_name,given_name,middle_name,suffix,prefix,degree.
Individual fields are created for each of these.
5. Provide examples on how to enrich HL7 coded values with actual contextual descriptions.
i.e. MSH-9 may have ADT^A08.
A new field Message_Type will also be present with a value of "Update Patient Information"
If the logs come from Cloverleaf's Interface engine there are additional extractions for thread_id, client ip, and client port.
Configuration
To use this add-on, manually configure the data input with the following input.conf
properties and the add-on will optimize the data automatically.
sourcetype = hl7_v2
These properties will also be utilized by the underlying searches and eventtypes .
The value from this Add-on is derived by it's ability to:
- Parse every HL7 message from log entry.*
- Extract every segment within any HL7 v2.x message into it's own Splunk Field.**
- Extract every field within every segment in the message.**
- Provide examples on how to extract values from HL7 subfields.
i.e. PID-5 contains family_name,given_name,middle_name,suffix,prefix,degree.
Individual fields are created for each of these. - Provide examples on how to enrich HL7 coded values with actual contextual descriptions.
i.e. MSH-9 may have ADT^A08.
A new field Message_Type will also be present with a value of "Update Patient Information"
If the logs come from Cloverleaf's Interface engine there are additional extractions for thread_id, client ip, and client port.
Notes:
*The datetime format can vary for each HL7 message. In lieu of using TIME_FORMAT inside props.conf
a custom datetime.xml was created inside this app. The extracted groups correspond to those specified
in the extracted attribute for define element. The groups (year, month, day, etc.) are specified in
Splunk's default datetime.xml in /etc/system/datetime.xml.
**Repeating Segments will be placed inside multi-value fields.
i.e. If there are four NTE segments in the message there will be one multi-value NTE field with four values.
The subsequent HL7 fields will only be extracted for the first segment in repeating segments.
Sample Files
The reference files used for this Add-on can be found in the cloverleaf_logs directory.
The samples directory contains files used to generate sample messages. If you add Splunk's Eventgen these samples will be replayed and put in the hl7 index. Please see https://github.com/splunk/eventgen and use the develop branch.
You'll also need to modify splunkUser and splunkPass in eventgen.conf to match credentials on your system.
Panels and Dashboards
This Add-On currently has prebuilt Panels and Dashboards.
Panels
HL7 - Events by Message Type - Over Time
HL7 - Event Count by Message Type - Over Time
HL7 - Raw Events - Over Time
Dashboards
Patient Activity History
TRANSACTION CONVERSION (assumes you also ingest the corresponding HL7 ACKs)
Special Thanks
To Davin Studer and The Vancouver Clinic that allowed extensive regression testing on this Add-On.
Release Notes
Version 1.09
-Updated Message Type Lookup ( hl7_v2_message_types.csv). Please use Message_Type_Description in lieu of previous Message_Type in your searches.
Version 1.08
- Updated files in samples to replay messages with missing patient names and identifiers.
- Put SEDCMD back to replace carriage returns with line feeds.
Version 1.07
- Removed indexes.conf
- Added instructions to README.md for extracting individual segments when they are repeating.
- Modified files in samples for eventgen to populate the default/main index instead of previously used hl7 index.
Version 1.06
- Modified LINE_BREAKER in props.conf to properly parse HL7 messages that contain "MSH" in the actual message, not just the three first characters.
- Commented out SEDCMD in props.conf for replacing carriage return with line feed.
- Added new sample file in cloverleaf_logs
Version 1.05
=== Splunk TA HL7 (Add-On) ===
Author: Joe Welsh - Splunk
Version/Date: 1.05 August 25, 2016
*Note: This app was previously Add-on for HL7
Supported product(s):
This add-on supports logs containing HL7 v2.X data.
Source type(s): This add-on will process data that is source-typed
as "hl7_v2".
Input requirements: N/A