This app will download email messages from an IMAP account/mailbox and index it into Splunk. Each email message will be treated as a separate splunk event.
This is an update to the "Splunk for IMAP" v1.20 app. It appears the Splunk employee who created the original app is not able to maintain it. Its last update was in 2009. So I decided (with permission of Splunk) to write an update! It cannot be named the same due to Splunk development naming rules, but this app can be considered the new version going forward; and thus started it at version 2.0. The original app will still work fine for Splunk v5 users and below.
You can help make IMAP Mailbox better by contributing: https://github.com/IMAPMailbox/IMAPmailbox.git
This section is to install on a centralized splunk setup. Look further down for distributed splunk design instructions.
By default, the IMAP app will create a new Splunk index named "mail". This is controlled by: default/indexes.conf. If you want the IMAP output to go to the default Splunk index:
Instructions if you are currently using Splunk for IMAP v1.20 and want to upgrade to this new app.
Install this app as instructions above, it will install in a new location than the original app.
Note: If you have any users with private saved searches, dashboards, alerts, then you will need to change their user app directory name so it appears for this app.
This app uses the same index "mail" as the original app, so all past indexed data will still exist.
For those who are running a distributed Splunk design or HA: ie separate forwarders, search heads, indexers, etc... Please follow these directions. Depending on your design YMMV. Please see this link for more instructions: [http://docs.splunk.com/Documentation/AddOns/released/Overview/Installingadd-ons]
Install this App on your Search head(s).
Disable the inputs: =- inputs.conf
[script://./bin/get_imap_email.py]
disabled = true
[script://.\bin\get_imap_email.py]
disabled = true
Disable the indexes: =- indexes.conf
[mail]
disabled = true
Install this App on your indexer(s).
Disable the inputs: =-inputs.conf
[script://./bin/get_imap_email.py]
disabled = true
[script://.\bin\get_imap_email.py]
disabled = true
Enable the index and replication: =- indexes.conf
[mail]
disabled = false
repFactor = auto
Disable app visibility: =- app.conf
[ui]
is_visible = false
Enable inputs on ONE of your systems.
Make sure that only ONE system is running the inputs for the IMAP app otherwise you will get email duplications. I personally recommend installing and running the inputs on a heavy forwarder.
It is recommeneded that your store your IMAP user password and splunk admin password encrypted in your local/imap.conf file.
source $SPLUNK_HOME/bin/setSplunkEnv
bin/genpass.sh
Message headers are indexed as key-value pairs, for example:
From = "User Name <user@emailcom>"
Subject = "This is sooo cool"
This makes it easy to generate reports from the email indexed in Splunk.
For example:
index=mail | top From
Also, note the quotes around the field values. This makes it easy to perform searches 'where' or regexes. For example, if you want to find all your email that was sent by any Will, do the following:
index=mail From="Will*"
Instead of searching with the "index=mail" tag as examples show above. I recommend using the macro imap_index
instead. If you change your index or even don't use one, this macro will still keep your searches working correctly.
If you have bugs or suggestions please contact pj@dysan.net.
What if your Exchange Admin does not want to enable imap? I suggest using the davmail server proxy, <http: davmail.sourceforge.net=""/>. It will convert IMAP requests to MAPI for exchange. You can load davmail on your local splunk server, or on any other server. Just point your imap.conf to the correct server and port. It will pass through the imap credentials to Exchange.
Some hints on troubleshooting:
Run the get imap script manually to look for mail issues.
sudo su - splunk
source $SPLUNK_HOME/bin/setSplunkEnv
$SPLUNK_HOME/bin/splunk cmd python bin/get_imap_email.py --debug
Check that the email message is plain text. Rich Text and HTML emails will not index by default inless you update imap.conf.
Google your error messages. :)
Duplicate Emails: If using a service like Gmail and you are getting a two copies or more of an email, this is normally because the email is being found twice on the server; once in the INBOX and the second in All Mail. Solution is to limit what folders you are checking. I suggest using:
This will download just your new email in your inbox and not the archived messages, which is why this is happening.
In my case I forward all unix root system messages to one mailbox that I index from. Then I can watch for errors and create alerts, such as cron failures.
I've signed up for mailing lists to my indexed email address and create reports and alerts based on only certain content I'm interested in looking for.
Just minor updates.
Just minor updates.
App completely updated for Splunk 6.0
# What's new in 2.0.5!
- Minor, very minor, updates.
- Updated documentation to support HA splunk design.
- Updated TA to support HA splunk design.
# What's new in 2.0.4!
- Added multi field support for email addresses, such as To, From, and Cc.
- Fixed format bug in the get_imap_email.py script.
# What's new in 2.0.3!
- Fixed a handler error in the setup file.
- Verified works with Splunk v6.2
# What's new in 2.0.2!
Just minor updates.
- Added a TA for distributed Splunk designs.
# What's new in 2.0.1!
Just minor updates.
- Added ui-prefs.conf to set default search times to 1 day instead of all time.
- Added a few more searches
- Renamed the searches to be more helpful.
## What's New in 2.0!
App completely updated for Splunk 6.0
- Updated nav bar
- New searches
- New dashboards
- Updated setup.xml
- Added Pivot Data Models
- Added app Icons
- Fixed some python bugs
- Fixed some genpass.sh bugs
- imap.conf changes to default settings:
* imapSearch
* deleteWhenDone
* useSSL
* port
Splunk AppInspect evaluates Splunk apps against a set of Splunk-defined criteria to assess the validity and security of an app package and components.
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 50GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.