Study Splunk

Want to learn about Splunk?, you came to the right spot ;) What does this blog contain so far?  What is Splunk? Splunk Enterprise Components? Installing Splunk? Installing Splunk Universal Forwarder? Walkthrough of Splunk Interface Search Modes Searching in Splunk #1 Splunk sub(commands) [top, rare, fields, table, rename, sort] #2 Splunk sub(commands) [eval, trim, chart, showperc, stats, avg] #3 Splunk sub(commands) [eval, round, trim, stats, ceil, exact, floor, tostring] #4 Splunk sub(commands) [timechart, geostats, iplocation] #5 Splunk sub(Commands) [sendemail, dedup, eval, concatenate, new_field] #6 Splunk sub(Commands) [fields, rename, replace, table, transaction] Bringing data into Splunk Bringing data into Splunk (Continued...) Enable receiving port on Splunk server Dealing with Time Still I am in a process of writing couple of more topics related to Splunk, but you can go thru any of the links given above !! Do let me know if you have...

What happened behind the hood? When we added the new log file to be monitored via the graphical interface, it created and added a configuration item into inputs.conf configuration file. You can find the configuration file here: $SPLUNK_HOME/etc/apps/search/local/inputs.conf You can also manually edit this file and add your custom stanza, once done to notify Splunk about the changes, the daemon needs to be restarted. It will contain something like this: [monitor:///var/log/messages] disabled = false host = splunk_server index = idx_messages sourcetype = linux_logs Above block is known as stanza, lets decipher this :)  monitor: This is used to specify which logfile(s) you want to monitor i.e. you can mention a specific logfile as well as full directory lets say you want to monitor everything under /var/log directory, just mention "monitor:///var/log/" and Splunk will try to index everything which is there in that directory. disabled: Lets say you w...

Now, lets dive deep into bringing data into Splunk. Splunk Enterprise can index any type of data, however it works best with data with timestamps. When Splunk indexes data, it breaks it into events based on timestamps. Every event or data which is indexed into Splunk should have a sourcetype ( helps to identify the type of data which is indexed ) assigned to it. In corporate environment, majorly forwarders ( ref: here ) are used to input data into Splunk but there are other ways as well in which you can get your data indexed to Splunk. Lets assume you want to monitor a log file of the local machine on which Splunk is installed then you can use the hyperlinks which are listed under "Local inputs" otherwise you can use the hyperlinks which are listed under "Forwarded inputs". For achieving that, you can navigate to "Settings" => "Data Inputs" => "Local Inputs" => "Add New" (NOTE: Make sure Splunk have a...

Walkthrough of Splunk Interface Accessible on port 8000 (default) Once installed, it will have some basic applications pre-installed. Contains a wide variety of hyperlinks/tabs to manage and play with you Splunk installation. If you need any sort of help, go to the help menu and there you can find couple of handy options like official documentation etc. Best part is, if you are stuck somewhere go to "Splunk Answers" and shoot your query. Splunk community is quite active and surely will help in getting your issue resolved.

Installing Splunk? Navigate to https://www.splunk.com Sign up for a free account on Splunk Login Click on "FREE SPLUNK" Select "Splunk Enterprise" Select the OS on which you want to install. Download the package. In my case I will in installing it on one of the AWS instance. So I will be choosing Linux 64 Bit .rpm. [root@ip ~]#  ls -lrt -rw-r--r-- 1 root root 345022297 Feb 28 07:14 splunk-7.2.4.2-fb30470262e3-linux-2.6-x86_64.rpm Here is my file  splunk-7.2.4.2-fb30470262e3-linux-2.6-x86_64.rpm. Create a user called splunk or whatever you want. Change its password. Give it sudo privileges. Install the rpm which we downloaded.  [splunk@ip opt]$ sudo rpm -ivh splunk-7.2.4.2-fb30470262e3-linux-2.6-x86_64.rpm We trust you have received the usual lecture from the local System Administrator. It usually boils down to these three things:     #1) Respect the privacy of others.     #2) Think before you type. ...

Splunk Enterprise Components? Search Head Basically its a graphical user interface and contains all the dashboards, charts etc. Also it enables us to have a solution to query the data according to our needs. Indexer It is the core component which do all the heavy tasks. Major task is to get the data parsed i.e. your data is broken down into events and stored in the indexer. Used by search head to query the data, once the data is queried all the events based on the search are returned back to the search head. Forwarders Universal For understanding sake you can say it as an agent. Collect data from remote data sources and feed it to Splunk indexer. Example: Flat files, logs (web-server, database). Very small daemon (light weight). Heavy Its heavier than universal forwarder. Additional capabilities of parsing and storing the data. Logically storing on heavy forwarder is not recommended. Parsing means masking of the data (removing secret information like pa...