This article will discuss the Splunk Search App and Splunk query language (SPL).
...
Search commands are streaming and non-streaming. Non-streaming commands are also dataset processing commands (e.g., ‘sort’ command), it just means it needs the entire dataset before processing the command
...
Distributable: streaming order of events don’t matter (e.g., ‘rex’ command which is Splunk’s regex command).
Centralized: order of events matter, that is, it transforms events in order.
Transforming: orders results into a dataset by transforming each event, statistical.
Generating: fetches event from indexes without any transformation, that is, it fetches data from the indexes and starts the search data stream, may be event or report generating.
...
The first command must be a ‘generating’ command. The commands in this primer are: ‘search’, ‘tstats’, and ‘makeresults’.
The results of one command are passed to another command with a '|' (pipe) symbol. This is streaming output of one command to the next. There can be (0…N) data processing commands.
The last segment is a method to “display” results visually or in a table.
...
The standard built-in fields for any Splunk deployment are: ‘host’, ‘source’, ‘sourcetype’, and ‘index’.
‘index’ isn't easy to remember at first,
‘sourcetype’ isn’t currently meaningful because we are sending only one event structure at this time with ‘sourcetype’=”_json” (JSON over https per the architecture diagram in the first article in this series).
The primary “generating” command is ‘search’, therefore it is the default and implied. To run a search,
Ensure the time range picker is set to “Last 30 minutes”
In the search bar, type in: search source=”*”; press enter, the search command is implied so the search bar runs the search and changes it to source=”*”.
Figure 4. shows the results of the search.
...
Element No. | Element | Description |
|---|---|---|
1 | Search bar | Specify your search criteria. |
2 | Time range picker | Specify the time period for the search, such as the last 30 minutes (default). |
3 | Search action buttons | Actions that can be performed on the search, primarily used for 2 purposes: working with a search job such as sending long running job to a background job; print or export results to CSV. |
4 | Search results tabs | The tab that your search results appear on depends on your search. Some searches produce a set of events, which appear on the Events tab. Other searches transform the data in events to produce search results, which appear on the Statistics tab. These are very important late for working with searches. |
5 | Timeline | A visual representation of the number of events that occur at each point in time. |
6 | Fields sidebar | Displays a list of the fields discovered in the events. The logic for fields to show in SELECTED FIELDS section is built-in fields and fields that appear in 100% of the population of events. The fields in INTERESTING FIELDS are fields extracted at index time that appear in the highest percentage of events in the results set. |
7 | 1 of 3 events displayed in results | This event has been expanded in the view pane just enough to inspect the ‘message’ field. Events are sent to Splunk as JSON over https and ‘message' field was not extracted during indexing and thus as in 'inner’ JSON contains important data to OpenMethods result sets. |
8 | 2 of 3 events displayed in results | This event was formatted by clicking the link associated with each event “Show as raw text”. The raw text, shown in another built-in field called ‘_raw’, is exactly how we sent it into Splunk from the MediaBar. |
9 | 3 of 3 events displayed in results | This JSON of this event was expanded out just enough to reveal some of the most important fields in our dataset: ‘crm’ data such as ‘crm.customer’, mediabar, ‘mb’ data such as ‘mb.agent’ and agent details, ‘mb.className’ and ‘mb.functionName’ which indicate which component/function/service this event might be related to (popflow, mediabar, request/respose from HIS, QA, CS). |
...
At the bottom of the Fields sidebar in Figure 3, there is a link “21 additional fields”. Upon clicking that it will open a dialog that shows the statistics for all fields.
The UI also supports mouse-over/hover. Hover over/click the ‘source’ field in the SELECTED FIELDS section and the view in Figure 5 below is shown.
 Figure 4. Statistics popup for ‘source’ | The statistics sample of the ‘source’ field shows ‘MediaBar’, ‘App Manager’, and ‘ConnectAPI PROD’. ‘MediaBar’ is the entire focus of these wiki articles. The other sources are in early stage. All events are by default sent to ‘index’=”main”, thus the baseline “Generating” command when should always include: index=”main” source=”MediaBar” (it is case-insensitive by default). |
...
We can use the Splunk search language to generate the statistics for ‘source’ ourselves.
| Code Block |
|---|
source="*" | top source source count percent MediaBar 229564 99.674790 App Manager 749 0.325210 230,313 events (8/10/20 8:35:00.000 PM to 8/10/20 9:05:00.000 PM Events (230,313) Statistics (2) |
Why is it different than the statistics for Figure 3 and Figure 4 above? We will answer that in a separate article.
...
Next: Quick Reference for Exploring OpenMethods Splunk Events