Log Monitoring

Log Monitoring detects log delivery outages on a per-index, per-sourcetype, per-host basis by tracking arrival time and diagnosing whether the forwarder agent or the source itself has stopped delivering data. The app uses scheduled searches to record the most recent arrival time for each source every five minutes and computes outage status at read time, eliminating the need for manual inventory maintenance. Detection is based on arrival time rather than event time to avoid false positives caused by clock skew or timestamp parsing failures. The app distinguishes between three failure modes: an agent that has stopped running, an agent that is running but not forwarding data, and a source that has genuinely stopped producing logs. This diagnosis relies on independent liveness signals from Splunk internal logs and deployment server phone-home data. All tracking data is stored in KV store collections and CSV lookups, with no custom indexes, Python code, scripted inputs, or custom commands required.