ITSI Module for Kafka smart monitoring
Overview
Details
READ THE UNIFIED GUIDE FOR KAFKA MONITORING: https://splunk-guide-for-kafka-monitoring.readthedocs.io
On-line documentation: https://da-itsi-telegraf-kafka.readthedocs.io
Install and configure events logging: https://splunkbase.splunk.com/app/4302
The ITSI module provides builtin and native monitoring for Apache Kafka components, as well as the Confluent stack components:
- Zookeeper
- Apache Kafka Brokers
- Apache Kafka Connect
- Confluent schema-registry
- Confluent ksql-server
- Confluent kafka-rest
- Kafka SLA and end to end monitoring with the Linkedin Kafka monitor
- Kafka Consumers lag monitoring with Burrow (Kafka Connect connectors, Kafka Streams, etc.)
Native ITSI integration:
- Builtin entities discovery for all components
- Services templates and KPI base searches
- Rich entity health views for each component of your Kafka environment
Multi-tenancy is fully supported by the application, relying on metrics tags support.
Flickr Gallery: https://flic.kr/s/aHsmkAJT9d
Docker testing template, get an environment for lab and demonstration up and ready in 30 seconds !
https://github.com/guilhemmarchand/kafka-docker-splunk
READ THE UNIFIED GUIDE FOR KAFKA MONITORING: https://splunk-guide-for-kafka-monitoring.readthedocs.io
Install and configure events logging: https://splunkbase.splunk.com/app/4302
Documentation: https://da-itsi-telegraf-kafka.readthedocs.io
Github repository: https://github.com/guilhemmarchand/DA-ITSI-TELEGRAF-KAFKA
Flickr Gallery: https://flic.kr/s/aHsmkAJT9d
Docker testing template, get an environment up and ready in 30 seconds !
https://github.com/guilhemmarchand/kafka-docker-splunk
Release Notes
Version 1.1.10
Version 1.1.10
- Fix: Avoid mcatalog calls with metric_name as metric_name, some versions of Splunk will incorrectly complain about this, and this causes appinpsect failures in Splunk Base
Version 1.1.9
Version 1.1.9
CAUTION: Zookeeper metric collection switches from Telegraf plugin to JMX for easier and more consistent metric collection across all the components.
If you were using the application prior to this version, please update your configuration to collect metrics via JMX.
See: https://telegraf-kafka.readthedocs.io/en/latest/implementation.html
- change: Zookeeper metric collection switches from Telegraf plugin to JMX collection via Jolokia Telegraf input
- fix: Topic entity dashboard should break by topic rather than per broker, fix aggregation when any selected
Version 1.1.8
CAUTION: Zookeeper metric collection switches from Telegraf plugin to JMX for easier and more consistent metric collection across all the components.
If you were using the application prior to this version, please update your configuration to collect metrics via JMX.
See: https://telegraf-kafka.readthedocs.io/en/latest/implementation.html
change: Zookeeper metric collection switches from Telegraf plugin to JMX collection via Jolokia Telegraf input
fix: Topic entity dashboard should break by topic rather than per broker, fix aggregation when any selected
Version 1.1.7
Version 1.1.7
- fix: Realtime traffic In and Out refer to the same field in Kafka broker entity view
Version 1.1.6
Version 1.1.6
- fix: Expose units for Zookeeper latency metrics in entity view
Version 1.1.5
Version 1.1.5
- fix: Static index reference in Kafka Brokers entities discovery report
- feature: Drilldown to single forms for Offline and Under-replicated partitions in Overview and Kafka Brokers entities views
Version 1.1.4
Version 1.1.4
- fix: incompatibility for ksql-server with latest Confluent release (5.1.x) due to metric name changes in JMX model
Version 1.1.3
Version 1.1.3
Burrow integration: Kafka Consumer Lag monitoring
- feature: New KPI basesearch and Service Template for Kafka Consumers Lag Monitoring with Burrow
- feature: New entity view for Kafka Consumers Lag monitoring
The Burrow integration provides advanced threshold less lag monitoring for Kafka Consumers, such as Kafka Connect connectors and Kafka Streams.
Version 1.1.1
Version 1.1.1
CAUTION: Breaking changes and major release, telegraf modification is required to provide global tags for env and label dimensions!
https://da-itsi-telegraf-kafka.readthedocs.io/en/latest/kafka_monitoring.html#telegraf-installation-and-configuration
Upgrade path:
- Upgrade telegraf configuration to provide the env and label tags
- Upgrade the module, manage entities and rebuild your services
release notes:
- fix: duplicated KPI id for topic/brokers under replicated replication leads in KPI rendering issues
- fix: entity rendering issue with Kafka SLA monitor health view
Version 1.1.0
Version 1.1.0:
CAUTION: Breaking changes and major release, telegraf modification is required to provide global tags for env and label dimensions!
- https://da-itsi-telegraf-kafka.readthedocs.io/en/latest/kafka_monitoring.html#telegraf-installation-and-configuration
**Upgrade path:**
- Upgrade telegraf configuration to provide the env and label tags
- Upgrade the module, manage entities and rebuild your services
**release notes:**
- feature: Support for multi-environments / multi-dc deployments with metrics tagging
- feature: Global rewrite of entities management and identification
- fix: Moved from second interval to cron schedule for entities import to avoid dup entities at addon installation time
- fix: Various fixes and improvements
Version 1.0.6
Version 1.0.6
- feature: Support for Confluent ksql-server
- feature: Support for Confluent kafka-rest
- feature: event logging integration with the TA-kafka-streaming-platform
Version 1.0.5
Version 1.0.5:
- feature: Support for Confluent schema-registry
- feature: Adding follower/leader info in Zookeeper entity view
Version 1.0.4
Version 1.0.4:
- fix: typo on partitions in Kafka brokers view
Version 1.0.3
Version 1.0.3:
- fix: missing entity filter in latency from Zookeeper view
- fix: incorrect static filter in state from Sink task view
Version 1.0.2
Version 1.0.2:
- fix: incorrect duration shown in Kafka Connect entity view
- feature: minor improvements in UIs
Version 1.0.1
Version 1.0.1
- fix: error in state of Source/Sink connector in dashboards