icon/x Created with Sketch.

Splunk Cookie Policy

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website. Learn more (including how to update your settings) here.
Accept Cookie Policy

Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading URL Toolbox
SHA256 checksum (url-toolbox_16.tgz) 0c78909074eee5275a4f44280ac1d2d27a218e60266c2395a618b3331836cc3b SHA256 checksum (url-toolbox_15.tgz) 597ad5b85ba7fb3e28232ff7463ba232cf770bda172e1c0f77c08a4074df938f SHA256 checksum (url-toolbox_14.tgz) 3e2983a842b6bf4807ffb3b10a5b1626b9a172830eba4e91d2040402f2d686e8 SHA256 checksum (url-toolbox_13.tgz) 2d59f75da5620063a672b1a81774d754e97731e6ec80a1f18be5196d8918e067 SHA256 checksum (url-toolbox_12.tgz) 6b3b45e09ae06e17dacb54611ff59c0471aae1c3732a23c3bf872855c7bbd309 SHA256 checksum (url-toolbox_11.tgz) 0e51b14cd93bee5f8b2f9ca5e29b8c099e8e89982c9e2048ee0fc0f0fbdbbd28
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

URL Toolbox

Admins: Please read about Splunk Enterprise 8.0 and the Python 2.7 end-of-life changes and impact on apps and upgradeshere.
Overview
Details
UTBox is a set of building blocks for Splunk specially created for URL manipulation.

UTBox has been created to be modular, easy to use and easy to deploy in any Splunk environments. It only needs to be deployed on Splunk Search Heads and the bundles will automatically be sent to your Splunk Indexers.

One of the core feature of UTBox is to correctly parse URLs and complicated TLDs (Top Level Domain) using the Mozilla Suffix List. Other functions like shannon entropy, counting, suites, meaning ratio, bayesian analysis, etc, are also available.

UTBox has firstly be created for security analysts but may fit other needs as it's a set of building blocks. Enterprise Security users will need to modify the import statement to use UTBox.


You should also take a look at URLParser for efficient URL parsing: https://splunkbase.splunk.com/app/3396/

Documentation

This tool has an embeded documentation located after installation in $SPLUNK_HOME/etc/apps/utbox/appserver/static/documentation.pdf

What is what ?

The syntax of a URL is as follow:

scheme://[user:password@]domain:port/path?query_string#fragment_id

Component details:

  • The scheme, which in many cases is the name of a protocol (but not always), defines how the resource will be obtained. Examples include http, https, ftp, file and many others.
  • The domain name or literal numeric IP address gives the destination location for the URL.
  • The port number, given in decimal, is optional; if omitted, the default for the scheme is used (80 for http, 443 for https, etc).
  • The path is used to specify and perhaps find the resource requested.
  • The query string contains data to be passed to software running on the server. It may contain name/value pairs separated by ampersands, for example ?first_name=John&last_name=Doe.
  • The fragment identifier, if present, specifies a part or a position within the overall resource or document. When used with HTML, it usually specifies a section or location within the page, and used in combination with Anchor elements or the "id" attribute of an element, the browser is scrolled to display that part of the page.

Source: http://en.wikipedia.org/wiki/Uniform_resource_locator

List of provided Lookups

For more information, please refer to the embeded documentation.

  • ut_parse_simple(url)
  • ut_parse(url, list) or ut_parse_extended(url, list)
  • ut_shannon(word)
  • ut_countset(word, set)
  • ut_suites(word, sets)
  • ut_meaning(word)
  • ut_bayesian(word)
  • ut_levenshtein(word1, word2)

Lookup & Macros

A generic lookup call in Splunk is of the format:

... | lookup <lookup_name> field AS field 

For example:

... | lookup ut_parse_simple_lookup url AS cs_uri

UTBox also provides macros definition for each lookup to make it easier to call the lookups. In the previous example, the call would be:

... | `ut_parse_simple(cs_uri)`

It is important to understand that those macros are simply shortcuts to lookups call. One can use one or another depending on their tastes.

History

  • v1.6, April 2016
    • new feature: the list parameter now accept a star (*) to load all lists (Mozilla, IANA, and Custom) in order to return the longest matching TLD.
    • Thanks to @davelugo for the idea!
  • v1.5, December 2015 Important changes
    • new feature: users can choose which list of TLD to load (2 provided by default, Mozilla Suffix List and IANA List)
    • ut_parse_extended requires now 2 arguments (url to parse and the list to use, ‘mozilla’, ‘iana' or 'custom')
    • ut_parse, mapped to ut_parse_extended requires the same.
  • v1.4, November 2015
    • Fix incorrect parsing for hosts having a port specified (ex: tcp://host.tld:443/)
  • v1.3, September 2015
    • Hosts are no more lowered (usefull when dealing with Base64 encoded data).
  • v1.2, May 2015
    • Added ut_levenshtein()
  • v1.1, May 2015
    • First public release

Release Notes

Version 1.6
April 18, 2016

Community Supported App

Version 1.5
Dec. 17, 2015

Version 1.4
Oct. 30, 2015

Version 1.3
Sept. 21, 2015

Version 1.2
May 12, 2015

Version 1.1
May 6, 2015

2,833
Installs
15,652
Downloads
Share Subscribe LOGIN TO DOWNLOAD

Subscribe Share

AppInspect Tooling

Splunk AppInspect evaluates Splunk apps against a set of Splunk-defined criteria to assess the validity and security of an app package and components.

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
© 2005-2019 Splunk Inc. All rights reserved.
Splunk®, Splunk>®, Listen to Your Data®, The Engine for Machine Data®, Hunk®, Splunk Cloud™, Splunk Light™, SPL™ and Splunk MINT™ are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners.