Bringing data into Splunk (Continued...)

-
What happened behind the hood?

When we added the new log file to be monitored via the graphical interface, it created and added a configuration item into inputs.conf configuration file.

You can find the configuration file here:
  • $SPLUNK_HOME/etc/apps/search/local/inputs.conf
  • You can also manually edit this file and add your custom stanza, once done to notify Splunk about the changes, the daemon needs to be restarted.
  • It will contain something like this:
[monitor:///var/log/messages]
disabled = false
host = splunk_server
index = idx_messages
sourcetype = linux_logs

Above block is known as stanza, lets decipher this :) 
  1. monitor: This is used to specify which logfile(s) you want to monitor i.e. you can mention a specific logfile as well as full directory lets say you want to monitor everything under /var/log directory, just mention "monitor:///var/log/" and Splunk will try to index everything which is there in that directory.
  2. disabled: Lets say you want to disable a monitor temporarily then you can mention true instead of false.
  3. host: A meaningful hostname, so that people can easily understand from where the events are coming in.
  4. index: Name of the index in which you want your data to be indexed.
  5. sourcetype: Type of data which is being indexed.
Note: When Splunk monitors the data, initially it index everything which it finds then it maintains a record of where it last read the data from so that if incase there is new data then it can index that as well.


Comments

Post a Comment

Popular posts from this blog

#3 Splunk sub(Commands) [eval, round, trim, stats, ceil, exact, floor, tostring]

-
Image
ROUND: Eval with round takes one or two numeric arguments, returning 'first value' rounded to the amount of decimal places specified in 'second value'.  By default it will remove all the decimals. Example: index=idx_messages sourcetype=linux_logs | eval new_rt= trim ( replace (response_time, "ms.", "")) | stats avg (new_rt) as Average | eval Average= round (Average) Remove all the decimal values. Example: | eval Average= round (Average,3) Rounded the value up-to 3 decimal places. Example: | eval Average= ceil (Average) Round the value up-to the next highest integer. Example: | eval Average= exact (Average) Give the output with maximum possible number of decimal values. Example: | eval Average= floor (Average) Round the value down to the nearest whole integer. Apart from this, there are other functions as well which are used by eval command, for instance pi (), sqrt () etc. TOSTRING: Hel
Read more

#6 Splunk sub(Commands) [fields, rename, replace, table, transaction]

-
FIELDS:  This command helps to keep or remove specified fields from the search results, below command will keep just three fields in your search result. Example: | fields request, rc, pt RENAME:  This command helps to rename field(s), below command will rename a field named as service to serviceType and RC as responseCode Example: | rename service AS serviceType, RC AS responseCode REPLACE:  This command helps to replace the values of fields with another value, below command will replace the values "fetchReport" and "viewReport" as "Report" in "serviceType' field. Example: | replace fetchReport with Report, viewReport with Report in serviceType TABLE: This command helps to format the results into tabular output. Example: | table request, rc, pt TRANSACTION:  This command helps to merge events into a single event based upon a common identifier, below command will create events based on two events i.e. it will fetch the txn-id w
Read more

#1 Splunk sub(Commands) [top, rare, fields, table, rename, sort]

-
Image
TOP: Will show you top results with respect to your field. Example: index=_internal | top limit=5 component RARE: Will help you to find out least common values of a field, i.e. it is similar to TOP but works in opposite direction. Example: index=_internal | rare limit=5 component FIELDS: Will help you to limit your columns, lets say you want to remove count from above table, fields can help you to achieve that. Though there are other usage of fields as well but you will learn slowly and gradually when you start building some complex queries. Example: index=_internal | top limit=5 component | fields component, percent TABLE: Same thing can be achieved via table as well. Example: index=_internal | top limit=5 component | table component, percent RENAME: Lets say you want to rename a column, for that you can use rename command. Example: index=_internal | top limit=5 component | rename percent AS percentage | t
Read more