Splunk Add-on for Microsoft Office 365 Reporting Web Service

NOTICE The functionality of this add-on has been incorporated into the support Splunk Add-on for Microsoft Office 365 https://splunkbase.splunk.com/app/4055

This add-on collects Message Trace data from Microsoft Office 365 including the following:

Sender
Recipient
Subject
To IP Address
From IP Address
Size
Date Received

About

Version: 2.0.0
Vendor Products: Microsoft Office 365 Reporting Message Trace Report
Splunk platform versions: 7.3 and later
Platforms: Platform independent
Visible: Yes, this add-on contains configuration

Prerequisites

Office 365 Subscription
An Azure AD Application Registration (if using OAuth)

The Microsoft API used by the add-on currently supports both Basic Authentication (username and password) and Modern Authentication (OAuth).

Minimum Permissions Needed in Office 365 for Modern Authentication (OAuth)

Create an Azure AD App Registration
Assign Azure AD roles to the application

The supported roles for the application to access the Reporting Web Service are:

Global Reader (recommended)
Global Administrator (not recommended)
Exchange Administrator (not recommended)

For general instructions about assigning roles in Azure AD, see View and assign administrator roles in Azure Active Directory.

Specify the permissions your app requires to access the Reporting Web Service
- In the Azure Portal, go to App registrations > All applications
- Select your application
- Select API Permissions
- Click Add a permission to display the Request API permission flyout page
- On the APIs my organization uses tab, select Office 365 Exchange Online
- On the flyout page, select Application permissions, and then click Add permissions
- The Reporting Web Service should now appear in the list of applications that your app requires permissions for
- Select Grant admin consent for "tenant name" to consent to the permissions given to your app

Minimum Permissions Needed in Office 365 for Basic Authentication

Note: Basic Authentication has been deprecated by Microsoft. Using Modern Authentication (OAuth) is recommended for data collection.

If using Basic Authentication, creating a new role for the user account with the following permissions is recommended:

Message Tracking
View-Only Audit Logs
View-Only Configuration
View-Only Recipients

Installing

Where to install this add-on

Unless otherwise noted, all supported add-ons can be safely installed to all tiers of a distributed Splunk platform deployment. See Where to install Splunk add-ons in Splunk Add-ons for more information.

This table provides a reference for installing this specific add-on to a distributed deployment of Splunk Enterprise.

Splunk platform component	Supported	Required	Comments
Search Heads	Yes	Yes	This add-on contains search-time knowledge. It is recommended to turn visibility off on your search heads to prevent data duplication errors that can result from running inputs on your search heads instead of (or in addition to) on your data collection node.
Heavy Forwarders	Yes	No (but recommended)	It is recommended to install this add-on on a heavy forwarder for data collection. Data collection should be configured in only 1 place to avoid duplicates.
Indexers	Yes	No	Not required as the parsing operations occur on the forwarders.
Universal Forwarders	No	No	Universal forwarders are not supported for data collection because the modular inputs require Python and the Splunk REST handler.

Configuration

Ensure the prerequisites are met above.

Access Splunk Web on the node of your Splunk platform installation that collects data for this add-on.
Launch the add-on, then click the Configuration menu item > Account > Add
Enter a name for the account configuration.
For Basic Authentication, enter the username and password to use for the add-on. Refer to the minimum permissions needed for the account above.
For Modern Authentication, enter a Client ID and Client Secret. Refer to the minimum permissions needed for the account above.
Select the Inputs menu item > Create New Input
Give the input a name.
Enter an interval (how often this input collects data).
Select an index.
Select an input mode.
Continuously Monitor will poll for new data based on the interval
Index Once requires a start date and end date and will only index the data in the given date range.
Select the account.
Specify the Query window size (minutes). When Continuously Monitor is selected, each time this input runs a start date is calculated for the Office 365 API query. The end date for the Office 365 API query will be the calculated start date plus the number of minutes specified by this parameter. For example, if the calculated start date is 2022-01-01T00:00:00 (midnight on January 1, 2022), the end date for the query will be 2022-01-01T00:01:00 (one hour after midnight) if the query window size is 60 minutes.
Specify the Delay throttle (minutes). Microsoft may delay trace events up to 24 hours and events are not guaranteed to be sequential during this delay ( reference ). This parameter specifies how close to "now" the end date for a query may be (where "now" is the time that the input runs). Continuing from the example above, if "now" is 2022-01-01T00:02:00 (two minutes after midnight) and the delay throttle is 60 minutes, the input will exit because the end date for the query is only 1 minute away from "now". Each time the input runs, the input will exit and do nothing until the end date is at least 60 minutes away from "now".
Select a Start date/time to specify how far back to go when initially collecting data. This parameter is optional when Continuously Monitor is selected, but the parameter is required when Index Once is selected.
Specify an End date/time if Index Once is selected.

Query Window Logic Diagram

Enabling Basic Authentication for a Single User in Your Environment

Note: This step is not needed if you are using Modern Authentication.

Connect to Exchange Online
From the Azure portal, start a cloud shell (PowerShell)

$Credential = Get-Credential
$ExchangeSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri "https://outlook.office365.com/powershell-liveid" -Credential $credential -Authentication "Basic" -AllowRedirection
Import-PSSession $ExchangeSession

Create/set the policy

New-AuthenticationPolicy -Name "Allow Basic Auth for Reporting Web Service"
Set-AuthenticationPolicy -Identity "Allow Basic Auth for Reporting Web Service" -AllowBasicAuthReportingWebServices:$true

Check the policy settings

Get-AuthenticationPolicy -Identity "Allow Basic Auth for Reporting Web Service" | fl AllowBasicAuth*

(You should see AllowBasicAuthReportingWebServices : True in the list returned)

Next, grant a specific user the created policy (the username is LegacyUser in this case):

Set-User -Identity LegacyUser -AuthenticationPolicy "Allow Basic Auth for Reporting Web Service"

Finally, check if all went well:

Get-User -Identity LegacyUser | fl auth*

You should see something like this:

AuthenticationPolicy : Allow Basic Auth for Reporting Web Service
Name : LegacyUser

CIM Compliance

Version 4.8.0

Email model

Release Notes

Version 2.0.0

Added support for Modern Authentication
jQuery updates

Version 1.2.0

Moved the username and password fields to the configuration tab in compliance with Splunk cloud.
Added support for Splunk 8

Version 1.1.0

Added controls for query window size and delay throttle
Added "Index Once" functionality
Added proxy support
Moved authentication parameters to the input to support multiple environments
Changed the default query start date from 7 to 29 days in the past due to O365 API changes
Bug fix for recursively gathering message traces

Version 1.0.0

Initial release

Privacy

Use of this add-on is permitted subject to your obligations, including data privacy obligations, under your agreement with Splunk and Splunk's Privacy Policy.

Contributors

Jason Conger
Luke Netto
Adam Balogh
Tetsuya Uemura
Lindon Morris