SXI Forum

SeanR

Administrator

Registered: 20-11-2018

Posts: 148

Elastic Stack

Below is a basic installation plan for Elastic Stack.  Elastic stack is made up of many components however this outlines the 3 main ones.

Basic configuration files can be found in the SXI Github Repository

• Elastic Search

• LogStach

• Kibana

Installation Steps:

1. To install some of these products as a Windows Service you will need the NSSM util which can be downloaded here

   • As a SXI standard we normally extract this util to X:\SXI\Utilities

2. Download the relevant Elastic Stack products from here

3. Extract each of the zip files into their own directory under the X:/SXI/Dashboard directory. 

   • X: is the drive you wish to run Elastic Stack from, Dashboard is a SXI standard location for the Elastic Installation.

   • There must be NO spaced in the path name to the elastic stack directories.

4. To install “elasticsearch” as a windows service perform the following tasks:

   1. Ensure that a System Variable called JAVA_HOME it pointing a jre directory that was installed previously

   2. Run the following command “elasticsearch-service.bat install” from within the X:\SXI\Dashboard\elasticsearch\bin directory

Elastic Search

To test is ElasticSearch was installed correctly start the service and browse to http://127.0.0.1:9200/ you should eventually be greeted with a json object which includes the following: 

"tagline" : "You Know, for Search"

Logstash

Logstash needs a config file to tell it what logs to collect from where.  These are normally kept in the config directory.  Please see Using Logstash for more information.

To install “logstash” as a windows service perform the following tasks:

1. Create the following dir x:\SXI\Dashboard\sincedb

2. Ensure you have created a config file as described in this post.

3. In X:\SXI\Dashboard\logstash\bin run the following command: “X:\SXI\Utilities\nssm.exe install logstash”

4. In the dialog box that pops up set Path: to X:\SXI\Dashboard\logstash\bin\logstash.bat

5. Startup directory is automatically populated with X:\SXI\Dashboard\logstash\bin

6. Set the Arguments to -f X:\SXI\Dashboard\logstash\config\name_of_the_config_you_created.json
           It may be better to use the -b X:\SXI\Dashboard\configs - this way we can keep all the logstash configs in one place and they will all be loaded

7. Set the Service name = Logstash 6.x where x = latest version available

8. Under the Details tab set Description: = Logstash 6.x Windows Service - https://elastic.co

Kibana

To install “kibana” as a windows service perform the following tasks:

1. In X:\SXI\Dashboard\kibana\bin run the following command: “X:\SXI\Utilities\nssm.exe install kibana”

2. In the dialog box that pops up set Path: to X:\SXI\Dashboard\kibana\bin\kibana.bat

3. Startup directory is automatically populated with X:\SXI\Dashboard\kibana\bin

4. Set the Service name = Kibana 6.x where x = latest version available

5. Under the Details tab set Description: = Kibana 6.x Windows Service - https://elastic.co

To test if Kibana was setup correctly start the service and browse to http://127.0.0.1:5601/ and you will eventually see the Kibana Icon.

SeanR

Administrator

Registered: 20-11-2018

Posts: 148

Re: Elastic Stack

Elastic Search Configuration

Basic configuration files can be found in the SXI Github Repository

Change elasticsearch.yml in X:\SXI\Dashboard\elasticsearch\config as follows

• Uncomment Line 37. ##path.logs: /path/to/logs and change to path.logs: X:/SXI/Dashboard/elasticsearch/logs

• Uncomment Line 56. #network.host: 192.168.0.1 and change 192.168.0.1 to point to hosts ip address

• Uncomment Line 59. #http.port: 9200 but leave the port as 9200

SeanR

Administrator

Registered: 20-11-2018

Posts: 148

Re: Elastic Stack

Kibana Configuration

Basic configuration files can be found in the SXI Github Repository

Change kibana.yml in X:\SXI\Dashboard\kibana\config as follows:

• Uncomment Line 2. but Leave the default port of 5601

• Uncomment Line 7. #server.host: "localhost" and point to your hosts ip address

• Uncomment Line 18. #server.name: "your-hostname" and insert your servers hostname

• Uncomment Line 22. #elasticsearch.url: "http://localhost:9200" and replace localhost with your hosts ip adress

SeanR

Administrator

Registered: 20-11-2018

Posts: 148

Re: Elastic Stack

Logstash Configuration

Basic configuration files can be found in the SXI Github Repository

NOTE: Do NOT change the logstash.yml config - leave the log level as is - this has a performance and capacity implication if this is changed.

• Create the following dir x:\SXI\Dashboard\sincedb

• Copy the SXI_Logstash_Config into the x:\SXI\Dashboard\configs directory

SeanR

Administrator

Registered: 20-11-2018

Posts: 148

Re: Elastic Stack

Metricbeat Installation and Configuration

Basic configuration files can be found in the SXI Github Repository

Metricbeat is use to send System metrics about the host where it is installed to an Elasticsearch server.  Metrics can be sent to Elasticsearch directly or through logstash.

1. Download MetricBeats from here and extract it to x:\SXI\Dashboard\metrics (this directory is not critical though as this can be installed anywhere)

2. Before installing metricbeat edit the install-service-metricbeat.ps1 and replace the service name with something more meaningful. E.g. "X-ServerMonitor"

3. To install the service open "PowerShell" as administrator and run the following:

   powershell -ExecutionPolicy ByPass -File install-service-metricbeat.ps1

   you should see the message that "X-ServerMonitor" was successfully installed.

4. Next run

   .\metricbeat.exe modules enable system

   which enables the operating system metrics to be collected.  there are a number of additional modules that whose explanation is beyond the scope of this post.

5. Running this:

   .\metricbeat.exe setup -e

   will send the default dashboard and visualizations to Kibana.

MarekR

Member

Registered: 21-02-2019

Posts: 19

Re: Elastic Stack

Remember when editing "install-service-metricbeat.ps1" to change displayName to X-ServerMonitor on line 13 and change line 14 as follows:

-path.data `"C:\SXI\Dashboard\metricbeat\Data`" -path.logs `"C:\SXI\Dashboard\metricbeat\logs`

Last edited by MarekR (04-03-2019 11:54:46)