SXI Forum

SeanR

Administrator

Registered: 20-11-2018

Posts: 148

Using Logstash

Logstash is used to write data into the Elastic Data Base from where it can be displayed using Kibana.

When configuring Logstash one needs to be mindful of the following:

The logstash log should only be run in TRACE when debugging new configuration. (Set the Logstash logging level in the logstash.yml file (..\Elk\logstash\config). Logging level property is "log.level: trace")

Ensure filters intended for a certain input type are not inadvertently run for different input type. This is done by defining a type in the input and using an "IF" with this type before applying the filter for example
       In the Input specify "type => "xpress""
       In the Filter specify "if [type] == "xpress"{.....}"

We should not write records that we will not be reporting on into Elastic. (During performance testing we noted when non significant records were being dumped into Elastic the performance was noticeably slower)

A set of Config files have been provided for SXI. (See the shared SXI_Logstash_Config folder on Google drive). All of these config files need to be placed into a folder pointed to by the -f switch. Logstash will read in all the .conf files. (It makes no difference to logstash if the config comes from one file or multiple files. Each file is really simple so this should reduce mistakes / frustration)

To configure new stuff we simply provide a extra file containing the appropriate input and a filter with match lines for any event you want to put into elastic. It might be prudent to have different files for XLayer and Docstore logs as these are produced with logback and are slightly different ie the times have milliseconds so transaction start and finish lapses will make sense.

When adding new config files you do not need to concern yourself with dropping unmatched events or adding things like the record_timestamp this is all done for you by the filter_XLayerCommon.conf config. We can add any other generic stuff to this when required. The only thing you need to ensure is that the "type" in the input matches the "if" in the filter. See XServiceBroker.conf.  This ensures that the filter is only run on the appropriate input. If this is omitted this filter will be run with all the inputs, and if the filter has numerous match lines this could be significant. You then simply add as many match lines as required.

I have also included a stdout output (currently commented) which is useful when adding additional matches to your config. It will display exactly what will go into elastic, but obviously you will need to run logstash from a command shell to see this.

Logstash can be run as a Daemon on nix platforms and a service on Windows platforms. To run from the command shell typically during development exercises run logstash.bat. Don't forget the -f switch to point to where you have saved the config files. (To get logstash to read from the beginning of a log file you need to delete the sincedb file. These have been configured to save in .../sxi/elk/sincedb/...)