EMC Isilon Add-on for Splunk Enterprise
Overview
Details
UPGRADE TA (v2.3 to v2.4.0)
Follow below steps to upgrade EMC Isilon Technology addon from version 2.3 to 2.4.0 (It is applicable only if you have configured Isilon oneFS server version 8.x or above)
1. Download tar of EMC Isilon Technology addon from splunk base (v2.4.0)
2. Take the backup of local directory residing in $SPLUNK_HOME/etc/apps/TA_EMC-Isilon
3. Delete the inputs.conf, isilonappsetup.conf, last_session_call_info.pos, passwords.conf files from the $SPLUNK_HOME/etc/apps/TA_EMC-Isilon/local directory
4. Extract tar of EMC Isilon Technology addon under $SPLUNK_HOME/etc/apps
5. Restart Splunk
6. Configure Add-on again from Manage-Apps > EMC Isilon Add-on for Splunk Enterprise > Set up
REQUIREMENTS
- Splunk version 6.5 or 6.6 (For Add-on version 2.3 onwards), Splunk version 6.2,6.3 or 6.4 (Till Add-on version 2.2).
- Isilon running OneFS version 7.1.x or above.
- If using a forwarder, it must be a Heavy Forwarder( because the universal forwarder does not include python).
- Splunk forwarder system should have 4 GB of RAM and a quad-core CPU to run this app smoothly.
- The forwarder system must have network access (HTTPs) to Isilon system which is to be splunked.
- Appropriate user credentials to collect data from Isilon node(Need to provide any one node ip from the Isilon cluster)
- Admin user ID and password for collecting data from Isilon node.
HOW TO INSTALL
- Add-On app can be installed through UI using "Manage Apps" or extract zip file directly into /opt/splunk/etc/apps/ folder.
HOW TO CONFIGURE
- Add-on version 2.3 onwards: After installation, go to the Apps->Manage Apps->Set up TA_EMC-Isilon. New set up screen will open, which will ask for Isilon node credentials. Provide ip address, username, password for any one Isilon node of your cluster(Please do not provide smart connector ip on setup screen. This TA expects actual ip of one of the node of Cluster). Also there are "Verify SSL Certificate?" and "SSL Certificate Path" fields. To execute API calls in secure way, User has to check "Verify SSL Certificate?" and provide certificate file path to "SSL Certificate Path" field. The certificate file must be stored on heavy forwarder. Also there is an option to provide index. Default value of index is isilon. User has to make sure, the provided index has already been created from menu Settings->Indexes. After providing these details, click save on Setup form. Once the setup has completed successfully, go to Settings->Advanced search->Search macros. Open entry 'isilon_index' from list and modify definition of macro according to index provided in setup form. The default definition is index=isilon.
- Till Add-on version 2.2: After installation, go to the Apps->Manage Apps->Set up TA_EMC-Isilon. New set up screen will open which will ask for Isilon node credentials. Provide ip address, username, password for any one Isilon node of your cluster and save them.(Please do not provide smart connector ip on setup screen. This TA expects actual ip of one of the node of Cluster)
- Splunk REST API will encrypt the password and store it in Add-on's folder itself in encrypted form, REST modular script will fetch these credentials through REST API to connect to the Isilon node.
- Restart the Splunk
- To enable forwarding syslog data from Isilon cluster, perform the following step:
- Make following changes in file /etc/mcp/override/syslog.conf on Isilon node (copy from /etc/mcp/default/syslog.conf if not present) :
- Put @<forwarders_ip_address> in front of the required log file and !* at the end of the syslog.conf file.
- Restart syslogd using this command - /etc/rc.d/syslogd restart.
- In some cases, syslog.conf file is already placed at /etc/mcp/override directory location but it is empty. In that case just put the log file name and the forwarder ip in that file.
- Below is the cotent of sample syslog.conf:
- Run the following commands to enable protocol, config and syslog auditing according to Isilon OneFS version:
- For EMC Isilon cluster with oneFS version 7.x.x -
- isi audit settings modify --protocol-auditing-enabled Yes
- isi audit settings modify --config-auditing-enabled Yes
- isi audit settings modify --config-syslog-enabled Yes
- For EMC Isilon cluster with oneFS version 8.x.x -
- isi audit settings global modify --protocol-auditing-enabled Yes
- isi audit settings global modify --config-auditing-enabled Yes
- isi audit settings global modify --config-syslog-enabled Yes
- isi audit settings modify --syslog-forwarding-enabled Yes
- Enable receiving the syslog data at forwarder. To do that, go to Settings -> Data Inputs -> UDP -> New. Provide port number(514 is recommended by Splunk), sourcetype as emc:isilon:syslog and index isilon to this data input entry.
- Make sure while receiving syslogs on you have set following metadata - index=isilon, sourcetype=emc:isilon:syslog.
auth. @<forwarders_ip_address>
!audit_conf
. @<forwarders_ip_address>
!audit_protocol
. @<forwarders_ip_address>
!
CIM Compatiblity
This app is compatible with "Authentication","Inventory" and "Performance" datamodels of Splunk CIM (Comman information model).
TEST INSTALLATION
The main app dashboard can take some time to populate the dashboards once data collection is started. A good test to see that you are receiving all of the data we expect is to run this search after several minutes:
search `isilon_index` | stats count by sourcetype
In particular, you should see these sourcetypes:
* emc:isilon:rest
* emc:isilon:syslog
If you don't see these sourcetypes, then
• For emc:isilon:rest, have a look at the messages for "emc:isilon:rest".
For Add-on version 2.3 onwards: User can see logs at $SPLUNK_HOME/var/log/isilon/emc_isilon.log file.
For Add-on version till 2.2: Here is a sample search that will show messages:
index=_internal component="ExecProcessor" "EMC Isilon Error:" | table _time host log_level message
• For "emc:isilon:syslog", check the syslog file in /etc/mcp/override/syslog.conf - it should have @<forwarders_ip_address> in front of the required log file and !* at the end of the syslog.conf file. Also run following command to see whether the syslog forwarding in enabled or not:
For EMC Isilon cluster with oneFS version 7.x.x - isi audit settings view
For EMC Isilon cluster with oneFS version 8.x.x - isi audit settings global view
EMC Isilon forward syslog and audit logs on 514 udp port by default. Please make sure port 514 is open and reserver for Isilon syslogs only.
UPGRADE Technology Add-on (From V2.2 to V2.3)
Follow below steps to upgrade EMC Isilon Technology addon from version 2.2 to 2.3
• Download tar of EMC Isilon Technology addon from splunk base (v2.3)
• Extract tar of EMC Isilon Technology addon under $SPLUNK_HOME/etc/apps
• Execute upgrade python script under $SPLUNK_HOME/etc/apps/TA_EMC-Isilon/bin/upgrade_from_v2.2_to_v2.3.py.
On execution, script will ask for input and user has to provide already setup nodes as comma separated value.
Command to execute script:
$SPLUNK_HOME/bin/splunk cmd python $SPLUNK_HOME/etc/apps/TA_EMC-Isilon/bin/upgrade_from_v2.2_to_v2.3.py
User can verify configured nodes from $SPLUNK_HOME/etc/apps/TA_EMC-Isilon/local/passwords.conf
This script will add stanza for each node, given as input while executing script, in file $SPLUNK_HOME/etc/apps/TA_EMC-Isilon/local/isilonappsetup.conf. Verify entry for each node in this file
• Restart Splunk
Release Notes
Version 2.4.0
Added support of new security patch coming in EMC Isilon cluster with oneFS version 8.1.0.4 and above.
Added support of pagination in active directory API calls.
Fixed 503 Server Error: Service Not Available Error for API calls.
Version 2.3
- Mandate HTTPS connection with OneFS with certificate verification.
- Removed indexes.conf from add-on - expecting user to create index manually and give same name while setup of Isilon node.
- Improved field extractions for audit logs of OneFS version(7.0.2.0, 8.0.0.1, 8.0..0.2, 8.0.0.4, 8.1.0.0)
- Changed error log locations to $SPLUNK_HOME/var/log/isilon/emc_isilon.log file
- Upgrade script needs to get executed while upgrading to version 2.3. More information in Overview section.
Version 2.2
Removed Input stanza for Syslog ingestion([udp://514]) and added instuction in documentation on how to setup syslog data ingestion.
Version: 2.1
- Added support for Active directory user & sid mapping through new REST API.
- Made changes in Isilon log parser to make it robust & generic for both Isilon version-7 and version-8 Security logs.
- Replaced Node_IP filter with Cluster_Name filter in existing Security logs dashboards.
- Security logs have been segregated in two categories - FileSystem Audit Logs and Authentication & Privilege activities.
- Added new dashboard to provide the user search facility for FS Audit logs across the Clusters.
Version 2.1
- Added support for Active directory user & sid mapping through new REST API.
- Made changes in Isilon log parser to make it robust & generic for both Isilon version-7 and version-8 Security logs.
- Replaced Node_IP filter with Cluster_Name filter in existing Security logs dashboards.
- Security logs have been segregated in two categories - FileSystem Audit Logs and Authentication & Privilege activities.
- Added new dashboard to provide the user search facility for FS Audit logs across the Clusters.
Version 2.0
-> Changed data collection method from Scripted Input to REST API Modular Input
-> Added support for Isilon Version 8.0
-> Added new data source(udp 514) and field extractions to integrate Isilion Syslogs & Audit logs
-> Added new API calls and extractions to comply with CIM models - Authentication, Inventory and Performance
How to upgrade the app
======================
Since the data collection method has been changed in release 2.0, user must need to remove the previous version of app ($SPLUNK_HOME/etc/apps/TA_EMC-Isilon) and perform the fresh installation of new bundle and set up the app again.Please note that removing old installation is not going to remove previously indexed data.