This page is a Quick Reference containing a punch list of run anywhere (copy/paste) Splunk searches to help explore the data in the OpenMethods Splunk events. The use of some advanced techniques is intentional and the foundational explanation for the techniques will be covered in other articles.
1. Basic Internal Structure
In this section, we’ll tackle Splunk’s own internal structure and how it manages indexes, sources, storage, event sizes, and types, and we’ll spot check the indexes _introspection and _internal.
1.1 What is the list of all indexes even if there is zero data in the index?
| eventcount summarize=false index=* index=_* | dedup index | rename count as countofevents | fields index countofevents | sort countofevents DESC |  |
Note: in order to conserve space on this page for the remaining queries,
- It is a best practice, when readability matters, to author Splunk queries per above with the pipe symbol as the first character of each new line. Instead, the queries will be compacted.
- Sample results will either not be included or will be converted to JSON and shown as a code block like this below (same results set as above).
[{"index":"1905405642","countofevents":"main"},{"index":"23781423","countofevents":"\"_audit\""},{"index":"10004574","countofevents":"\"_internal\""},{"index":"1755528","countofevents":"\"_introspection\""},{"index":"85111","countofevents":"summary"},{"index":"24343","countofevents":"\"_telemetry\""},{"index":"4","countofevents":"lastchanceindex"},{"index":"0","countofevents":"\"_thefishbucket\""},{"index":"0","countofevents":"demomatricindex"},{"index":"0","countofevents":"history"},{"index":"0","countofevents":"netaddinsimport"},{"index":"0","countofevents":"popflowscriptindex"},{"index":"0","countofevents":"tunnelmetricsindex"}]
list of data size, usage, average no. of bytes, events, event size of _raw and license index
2. Basic OpenMethods Topology
In this section, we’ll discover broadly where/how to look for events that give an overall view of how our software is deployed and being used.
2.1 List of customers and their CTI environment/location of agents (or at least URL agents use to access HIS)?
index="main" earliest="-4h" | stats values(network.his.uri) as hisurl, values(network.his.model) as hiscti by crm.customer
2.2 List of customers and their sites/versions?
| tstats count WHERE (index="main" earliest=-5d@d latest=now source="mediabar") BY crm.customer mb.version network.appManagerDomain network.crmHost | sort mb.version DESC | dedup network.crmHost | table crm.customer mb.version network.crmHost | sort crm.customer
2.3 How to segment agent usage by production versus lower environments?
| tstats count WHERE (index="main" earliest="-1h" [|inputlookup spl-customer-host.csv | where cloudenv="prod" | fields displaycustomer hostlookup | lookup spl-customer-host.csv displaycustomer cloudenv OUTPUT hostlookup | fields - displaycustomer | rename hostlookup as host | format]) BY _time crm.username host | stats dc(crm.username) as total_User by host
The Splunk ‘lookup’ data structure that made the above query possible:
| inputlookup spl-customer-host.csv
| WHERE NOT (displaycustomer in ("omdemo","omdev", "omqa", "omtrain"))
| dedup displaycustomer
| lookup spl-customer-host.csv displaycustomer OUTPUT crmcustomer cloudenv hostlookup
2.4 How to convert Splunk events to look like basic log statements I am used to for troubleshooting?
| tstats count WHERE (index="main" earliest="8/1/2020:06:00:00" latest="8/1/2020:06:30:00" source="mediabar" host="https://chewy.custhelp.com" ) BY _time logLevel crm.instanceId crm.groupId crm.id mb.className mb.functionName message span=1s