JellyFisher
Overview
Details
JellyFisher is a Splunk custom search command that leverage the excellent jellyfish python's library to do approximate and phonetic strings matching.
String comparison algorithms:
- Levenshtein Distance
- Damerau-Levenshtein Distance
- Jaro Distance
- Jaro-Winkler Distance
- Match Rating Approach Comparison
- Hamming Distance
Phonetic encoding algortihms:
- American Soundex
- Metaphone
- NYSIIS (New York State Identification and Intelligence System)
- Match Rating Codex
Example:
- The Levenshtein distance between "kitten" and "sitting" is 3.
- The soundex representation of both "Robert" and "Rupert" is "R163"
String comparison algorithms:
- Levenshtein Distance
- Damerau-Levenshtein Distance
- Jaro Distance
- Jaro-Winkler Distance
- Match Rating Approach Comparison
- Hamming Distance
Phonetic encoding algortihms:
- American Soundex
- Metaphone
- NYSIIS (New York State Identification and Intelligence System)
- Match Rating Codex
Example:
- The Levenshtein distance between "kitten" and "sitting" is 3.
- The soundex representation of both "Robert" and "Rupert" is "R163"
Usage
... | jellyfisher algorithm_name(fields_to_read_words_from)Results will be written into a new field of the name of the chosen algorithm (ex: levensthein_distance)
Examples
... | jellyfisher levensthein_distance(sourcetype,source)
... | jellyfisher jaro_distance(user,sourcetype)
... | jellyfisher soundex(sourcetype)Algorithms list
distance alogrithms:
- levenshtein_distance
- damerau_levenshtein_distance
- jaro_distance
- jaro_winkler
- match_rating_comparison
- hamming_distance
phonetic alogrithms:
- soundex
- nysiis
- match_rating_codex
- metaphone
- porter_stem
Inputlookup
The following search is given as demonstration purpose to load a bunch of words (process names) from a CSV lookup file and then compute their distances using the map() command. This is not the perfect implementation, but a good example to start from :)
| inputlookup processNames.csv
| map search="search index=windows | head 10 | stats count by process | eval pName = $processName$ | jellyfisher levenshtein_distance($processName$, process)"
| table process, pName, levenshtein_distanceRelease Notes
Version 0.1.0
June 21, 2017