We will be installing,configuring elasticsearch and kibana 8.4, but steps should be same for most versions.
Our cluster will have 3 master nodes, 3 hot data nodes, 3 warm data nodes and 1 machine learning node.
1) pre-requisites
1a) create /var/lib/elasticsearch mount point on all the nodes.
1b) turn off swap on OS(to ensure JVM heap is not swapped out).
1c) since we are using packages to install elasticsearch, ulimits are enforced in systemd unit file /usr/lib/systemd/system/elasticsearch.service.
1d) settings like file descriptors, max processes, max virtual memory size , max file size, etc are controlled from the systemd unit file.
1e) change default value of TCP retransmission timeout value, update the net.ipv4.tcp_retries2 setting in /etc/sysctl.conf to 5, and sysctl -w net.ipv4.tcp_retries2=5.
2) Installing elasticsearch
Our cluster will have 3 master nodes, 3 hot data nodes, 3 warm data nodes and 1 machine learning node.
2a) Import elasticsearch PGP key.
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
2b) Install apt-transport-https package
sudo apt install apt-transport-https
2c) save the repo,
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
2d) update the repo and install the package,
apt update && apt install elasticsearch
apt-get install elasticsearch
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
elasticsearch
0 upgraded, 1 newly installed, 0 to remove and 119 not upgraded.
Need to get 0 B/566 MB of archives.
After this operation, 1,170 MB of additional disk space will be used.
Selecting previously unselected package elasticsearch.
(Reading database ... 111616 files and directories currently installed.)
Preparing to unpack .../elasticsearch_8.4.3_amd64.deb ...
Creating elasticsearch group... OK
Creating elasticsearch user... OK
Unpacking elasticsearch (8.4.3) ...
Setting up elasticsearch (8.4.3) ...
--------------------------- Security autoconfiguration information ------------------------------
Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.
The generated password for the elastic built-in superuser is : B25meUI2L6WcfTWBNvNp
If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.
You can complete the following actions at any time:
Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.
Generate an enrollment token for Kibana instances with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.
Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.
2e) Ansible playbook to install the package.
---
- hosts: elasticsearch
become: true
gather_facts: true
tasks:
- name: Import the Elasticsearch PGP key
apt_key:
url: https://artifacts.elastic.co/GPG-KEY-elasticsearch
keyring: /usr/share/keyrings/elasticsearch-keyring.gpg
state: present
- name: Install apt-transport-https
apt:
name: apt-transport-https
state: present
# Add elasticsearch repo into sources list file /etc/apt/sources.list.d/elastic-8.x.list, after adding it will also run apt update or apt-get update by default
- apt_repository:
repo: 'deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main'
state: present
filename: elastic-8.x.list
- name: Install a specific version of elasticsearch
apt:
name: elasticsearch=8.4.3
state: present
update_cache: yes
2f) enable the service to start automatically on boot
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
3) Generating certificates to enable TLS for transport and http.
3a) Generate CA certificate.
Login to one of the node where you installed elasticsearch , and issue below command to generate CA certificate. For higher protection, ensure you are setting password the certificate when it prompts below at the end and ensure you save that password in a secure location to use it later.
/usr/share/elasticsearch/bin/elasticsearch-certutil ca --out /root/elasticsearch_certs/elasticsearch-test-ca.p12
/usr/share/elasticsearch/bin/elasticsearch-certutil ca --out /root/elasticsearch_certs/elasticsearch-test-ca.p12
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.
The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.
Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority
By default the 'ca' mode produces a single PKCS#12 output file which holds:
* The CA certificate
* The CA's private key
If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key
Enter password for elasticsearch-test-ca.p12 :
root@jumperserver:~/elasticsearch_certs# ls
elasticsearch-test-ca.p12
3b) Generate node certificates
We use node certificates to join nodes to cluster and for transport layer encrytion. add all of your node details with dns name and ip into an yaml file like below,
root@jumperserver:~# cat /root/elasticsearch_certs/instances.yaml
instances:
- name: "test-elastic-master01"
ip: "10.10.4.6"
dns: "test-elastic-master01"
- name: "test-elastic-master02"
ip: "10.10.4.7"
dns: "test-elastic-master02"
- name: "test-elastic-master03"
ip: "10.10.4.8"
dns: "test-elastic-master03"
- name: "test-elastic-hotdata01"
ip: "10.10.4.2"
dns: "test-elastic-hotdata01"
- name: "test-elastic-hotdata02"
ip: "10.10.4.3"
dns: "test-elastic-hotdata02"
- name: "test-elastic-hotdata03"
ip: "10.10.4.4"
dns: "test-elastic-hotdata03"
- name: "test-elastic-warmdata01"
ip: "10.10.4.11"
dns: "test-elastic-warmdata01"
- name: "test-elastic-warmdata02"
ip: "10.10.4.12"
dns: "test-elastic-warmdata02"
- name: "test-elastic-warmdata03"
ip: "10.10.4.13"
dns: "test-elastic-warmdata03"
- name: "test-elastic-ml01"
ip: "10.10.4.10"
dns: "test-elastic-ml01"
below you need to enter CA certificate password that you entered in step 3a, and ensure you set password for each and every node certificate ( you can set same password for all the nodes or different password as per security compliance)
/usr/share/elasticsearch/bin/elasticsearch-certutil cert --in /root/elasticsearch_certs/instances.yaml --out /root/elasticsearch_certs/server-cert-bundle.zip --ca /root/elasticsearch_certs/elasticsearch-test-ca.p12
root@elasticsearch-jumperserver:~/elasticsearch_certs# /usr/share/elasticsearch/bin/elasticsearch-certutil cert --in /root/elasticsearch_certs/instances.yaml --out /root/elasticsearch_certs/server-cert-bundle.zip --ca /root/elasticsearch_certs/elasticsearch-test-ca.p12
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.
The 'cert' mode generates X.509 certificate and private keys.
* By default, this generates a single certificate and key for use
on a single instance.
* The '-multiple' option will prompt you to enter details for multiple
instances and will generate a certificate and key for each one
* The '-in' option allows for the certificate generation to be automated by describing
the details of each instance in a YAML file
* An instance is any piece of the Elastic Stack that requires an SSL certificate.
Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
may all require a certificate and private key.
* The minimum required value for each instance is a name. This can simply be the
hostname, which will be used as the Common Name of the certificate. A full
distinguished name may also be used.
* A filename value may be required for each instance. This is necessary when the
name would result in an invalid file or directory name. The name provided here
is used as the directory name (within the zip) and the prefix for the key and
certificate files. The filename is required if you are prompted and the name
is not displayed in the prompt.
* IP addresses and DNS names are optional. Multiple values can be specified as a
comma separated string. If no IP addresses or DNS names are provided, you may
disable hostname verification in your SSL configuration.
* All certificates generated by this tool will be signed by a certificate authority (CA)
unless the --self-signed command line option is specified.
The tool can automatically generate a new CA for you, or you can provide your own with
the --ca or --ca-cert command line options.
By default the 'cert' mode produces a single PKCS#12 output file which holds:
* The instance certificate
* The private key for the instance certificate
* The CA certificate
If you specify any of the following options:
* -pem (PEM formatted output)
* -multiple (generate multiple certificates)
* -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files
Enter password for CA (/root/elasticsearch_certs/elasticsearch-test-ca.p12) :
Enter password for test-elastic-master01/test-elastic-master01.p12 :
Enter password for test-elastic-master02/test-elastic-master02.p12 :
Enter password for test-elastic-master03/test-elastic-master03.p12 :
Enter password for test-elastic-hotdata01/test-elastic-hotdata01.p12 :
Enter password for test-elastic-hotdata02/test-elastic-hotdata02.p12 :
Enter password for test-elastic-hotdata03/test-elastic-hotdata03.p12 :
Enter password for test-elastic-warmdata01/test-elastic-warmdata01.p12 :
Enter password for test-elastic-warmdata02/test-elastic-warmdata02.p12 :
Enter password for test-elastic-warmdata03/test-elastic-warmdata03.p12 :
Enter password for test-elastic-ml01/test-elastic-ml01.p12 :
Certificates written to /root/elasticsearch_certs/server-cert-bundle.zip
This file should be properly secured as it contains the private keys for
all instances
After unzipping the file, there will be a directory for each instance.
Each instance has a single PKCS#12 (.p12) file containing the instance
certificate, instance private key and the CA certificate
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.
For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
root@elasticsearch-jumperserver:~/elasticsearch_certs#
below we are checking the generated certificates,
root@elasticsearch-jumperserver:~/elasticsearch_certs# ls
elasticsearch-test-ca.p12 instances.yaml server-cert-bundle.zip
root@elasticsearch-jumperserver:~/elasticsearch_certs#
root@elasticsearch-jumperserver:~/elasticsearch_certs# unzip server-cert-bundle.zip
Archive: server-cert-bundle.zip
creating: test-elastic-master01/
inflating: test-elastic-master01/test-elastic-master01.p12
creating: test-elastic-master02/
inflating: test-elastic-master02/test-elastic-master02.p12
creating: test-elastic-master03/
inflating: test-elastic-master03/test-elastic-master03.p12
creating: test-elastic-hotdata01/
inflating: test-elastic-hotdata01/test-elastic-hotdata01.p12
creating: test-elastic-hotdata02/
inflating: test-elastic-hotdata02/test-elastic-hotdata02.p12
creating: test-elastic-hotdata03/
inflating: test-elastic-hotdata03/test-elastic-hotdata03.p12
creating: test-elastic-warmdata01/
inflating: test-elastic-warmdata01/test-elastic-warmdata01.p12
creating: test-elastic-warmdata02/
inflating: test-elastic-warmdata02/test-elastic-warmdata02.p12
creating: test-elastic-warmdata03/
inflating: test-elastic-warmdata03/test-elastic-warmdata03.p12
creating: test-elastic-ml01/
inflating: test-elastic-ml01/test-elastic-ml01.p12
root@elasticsearch-jumperserver:~/elasticsearch_certs#
root@elasticsearch-jumperserver:~/elasticsearch_certs# ls
elasticsearch-test-ca.p12 test-elastic-hotdata02 test-elastic-master01 test-elastic-master03 test-elastic-warmdata01 test-elastic-warmdata03 server-cert-bundle.zip
test-elastic-hotdata01 test-elastic-hotdata03 test-elastic-master02 test-elastic-ml01 test-elastic-warmdata02 instances.yaml
root@elasticsearch-jumperserver:~/elasticsearch_certs#
root@elasticsearch-jumperserver:~/elasticsearch_certs# ls -ltr *
-rw-r--r-- 1 root root 876 Oct 26 18:49 instances.yaml
-rw------- 1 root root 2672 Oct 26 18:55 elasticsearch-test-ca.p12
-rw------- 1 root root 39406 Oct 26 18:56 server-cert-bundle.zip
test-elastic-master01:
total 4
-rw-r--r-- 1 root root 3700 Oct 26 18:56 test-elastic-master01.p12
test-elastic-master02:
total 4
-rw-r--r-- 1 root root 3700 Oct 26 18:56 test-elastic-master02.p12
test-elastic-master03:
total 4
-rw-r--r-- 1 root root 3700 Oct 26 18:56 test-elastic-master03.p12
test-elastic-hotdata01:
total 4
-rw-r--r-- 1 root root 3702 Oct 26 18:56 test-elastic-hotdata01.p12
test-elastic-hotdata03:
total 4
-rw-r--r-- 1 root root 3702 Oct 26 18:56 test-elastic-hotdata03.p12
test-elastic-hotdata02:
total 4
-rw-r--r-- 1 root root 3702 Oct 26 18:56 test-elastic-hotdata02.p12
test-elastic-warmdata01:
total 4
-rw-r--r-- 1 root root 3704 Oct 26 18:56 test-elastic-warmdata01.p12
test-elastic-warmdata03:
total 4
-rw-r--r-- 1 root root 3704 Oct 26 18:56 test-elastic-warmdata03.p12
test-elastic-warmdata02:
total 4
-rw-r--r-- 1 root root 3704 Oct 26 18:56 test-elastic-warmdata02.p12
test-elastic-ml01:
total 4
-rw-r--r-- 1 root root 3676 Oct 26 18:56 test-elastic-ml01.p12
root@elasticsearch-jumperserver:~/elasticsearch_certs#
3c) Generate http certificate.
generate http certificates for http encryption, ensure you enter hostnames and ip's of the machines from which you would like you to communicate with elaticsearch over http, e.g jumpservers, kibana, elasticsearch nodes , so on.
/usr/share/elasticsearch/bin/elasticsearch-certutil http
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# /usr/share/elasticsearch/bin/elasticsearch-certutil http
## Elasticsearch HTTP Certificate Utility
The 'http' command guides you through the process of generating certificates
for use on the HTTP (Rest) interface for Elasticsearch.
This tool will ask you a number of questions in order to generate the right
set of files for your needs.
## Do you wish to generate a Certificate Signing Request (CSR)?
A CSR is used when you want your certificate to be created by an existing
Certificate Authority (CA) that you do not control (that is, you don't have
access to the keys for that CA).
If you are in a corporate environment with a central security team, then you
may have an existing Corporate CA that can generate your certificate for you.
Infrastructure within your organisation may already be configured to trust this
CA, so it may be easier for clients to connect to Elasticsearch if you use a
CSR and send that request to the team that controls your CA.
If you choose not to generate a CSR, this tool will generate a new certificate
for you. That certificate will be signed by a CA under your control. This is a
quick and easy way to secure your cluster with TLS, but you will need to
configure all your clients to trust that custom CA.
Generate a CSR? [y/N]n
## Do you have an existing Certificate Authority (CA) key-pair that you wish to use to sign your certificate?
If you have an existing CA certificate and key, then you can use that CA to
sign your new http certificate. This allows you to use the same CA across
multiple Elasticsearch clusters which can make it easier to configure clients,
and may be easier for you to manage.
If you do not have an existing CA, one will be generated for you.
Use an existing CA? [y/N]y
## What is the path to your CA?
Please enter the full pathname to the Certificate Authority that you wish to
use for signing your new http certificate. This can be in PKCS#12 (.p12), JKS
(.jks) or PEM (.crt, .key, .pem) format.
CA Path: /root/elasticsearch_certs/elasticsearch-test-ca.p12
Reading a PKCS12 keystore requires a password.
It is possible for the keystore's password to be blank,
in which case you can simply press <ENTER> at the prompt
Password for elasticsearch-test-ca.p12:
## How long should your certificates be valid?
Every certificate has an expiry date. When the expiry date is reached clients
will stop trusting your certificate and TLS connections will fail.
Best practice suggests that you should either:
(a) set this to a short duration (90 - 120 days) and have automatic processes
to generate a new certificate before the old one expires, or
(b) set it to a longer duration (3 - 5 years) and then perform a manual update
a few months before it expires.
You may enter the validity period in years (e.g. 3Y), months (e.g. 18M), or days (e.g. 90D)
For how long should your certificate be valid? [5y] 10y
## Do you wish to generate one certificate per node?
If you have multiple nodes in your cluster, then you may choose to generate a
separate certificate for each of these nodes. Each certificate will have its
own private key, and will be issued for a specific hostname or IP address.
Alternatively, you may wish to generate a single certificate that is valid
across all the hostnames or addresses in your cluster.
If all of your nodes will be accessed through a single domain
(e.g. node01.es.example.com, node02.es.example.com, etc) then you may find it
simpler to generate one certificate with a wildcard hostname (*.es.example.com)
and use that across all of your nodes.
However, if you do not have a common domain name, and you expect to add
additional nodes to your cluster in the future, then you should generate a
certificate per node so that you can more easily generate new certificates when
you provision new nodes.
Generate a certificate per node? [y/N]N
## Which hostnames will be used to connect to your nodes?
These hostnames will be added as "DNS" names in the "Subject Alternative Name"
(SAN) field in your certificate.
You should list every hostname and variant that people will use to connect to
your cluster over http.
Do not list IP addresses here, you will be asked to enter them later.
If you wish to use a wildcard certificate (for example *.es.example.com) you
can enter that here.
Enter all the hostnames that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.
test-elastic-master01
test-elastic-master02
test-elastic-master03
test-elastic-kibana01
test-elastic-clustmon01
elasticsearch-jumpserver
You entered the following hostnames.
- test-elastic-master01
- test-elastic-master02
- test-elastic-master03
- test-elastic-kibana01
- test-elastic-clustmon01
- elasticsearch-jumpserver
Is this correct [Y/n]Y
## Which IP addresses will be used to connect to your nodes?
If your clients will ever connect to your nodes by numeric IP address, then you
can list these as valid IP "Subject Alternative Name" (SAN) fields in your
certificate.
If you do not have fixed IP addresses, or not wish to support direct IP access
to your cluster then you can just press <ENTER> to skip this step.
Enter all the IP addresses that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.
10.10.4.6
10.10.4.7
10.10.4.8
10.10.4.5
10.10.4.16
10.10.4.17
10.10.4.18
10.10.4.1
10.10.4.31
You entered the following IP addresses.
- 10.10.4.6
- 10.10.4.7
- 10.10.4.8
- 10.10.4.5
- 10.10.4.16
- 10.10.4.17
- 10.10.4.18
- 10.10.4.1
- 10.10.4.31
Is this correct [Y/n]Y
## Other certificate options
The generated certificate will have the following additional configuration
values. These values have been selected based on a combination of the
information you have provided above and secure defaults. You should not need to
change these values unless you have specific requirements.
Key Name: test-elastic-master01
Subject DN: CN=test-elastic-master01
Key Size: 2048
Do you wish to change any of these options? [y/N]N
## What password do you want for your private key(s)?
Your private key(s) will be stored in a PKCS#12 keystore file named "http.p12".
This type of keystore is always password protected, but it is possible to use a
blank password.
If you wish to use a blank password, simply press <enter> at the prompt below.
Provide a password for the "http.p12" file: [<ENTER> for none]
Repeat password to confirm:
## Where should we save the generated files?
A number of files will be generated including your private key(s),
public certificate(s), and sample configuration options for Elastic Stack products.
These files will be included in a single zip archive.
What filename should be used for the output zip file? [/usr/share/elasticsearch/elasticsearch-ssl-http.zip]
Zip file written to /usr/share/elasticsearch/elasticsearch-ssl-http.zip
root@elasticsearch-jumperserver:~/elasticsearch_certs/http#
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# ls
elasticsearch elasticsearch-ssl-http.zip kibana
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# mv elasticsearch-ssl-http.zip elasticsearch-ssl-http.zip_old
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# mv elasticsearch elasticsearch_old
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# mv kibana kibana_old
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# pwd
/root/elasticsearch_certs/http
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# cp /usr/share/elasticsearch/elasticsearch-ssl-http.zip .
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# unzip elasticsearch-ssl-http.zip
Archive: elasticsearch-ssl-http.zip
creating: elasticsearch/
inflating: elasticsearch/README.txt
inflating: elasticsearch/http.p12
inflating: elasticsearch/sample-elasticsearch.yml
creating: kibana/
inflating: kibana/README.txt
inflating: kibana/elasticsearch-ca.pem
inflating: kibana/sample-kibana.yml
root@elasticsearch-jumperserver:~/elasticsearch_certs/http#
root@elasticsearch-jumperserver:~/elasticsearch_certs/http# ls
elasticsearch elasticsearch_old elasticsearch-ssl-http.zip elasticsearch-ssl-http.zip_old kibana kibana_old
root@elasticsearch-jumperserver:~/elasticsearch_certs/http#
4) Copy the generated certificates
Copy the node certificate and http certificate to respective nodes to the path /etc/elasticsearch/certs/
Note: Node certificate is different for each and every elasticsearch node, http certificate is common for all the nodes.
*5) Setting keystore and trustore for transport and http *
Transport Truststore password is the password of CA certificate.
Transport Keystore password is the password of node certificates.
Transport http password is the password of http certificate.
set transport truststore/keystore and http keystore with below commands on all the nodes, you need run below commands on each and every elasticsearch nodes,
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
6) Configuring elasticsearch parameters
Setting the configuration in /etc/elasticsearch/elasticsearch.yml, comment all the existing lines and append below after changing ip and hostnames to your node ip's and hostnames,
6a) Master nodes
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
cluster.name: test-elasticsearch
node.name: test-elastic-master01
network.host: 10.10.4.6
discovery.seed_hosts: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
cluster.initial_master_nodes: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
node.roles: [ master ]
xpack.watcher.enabled: true
# transport SSL/TLS
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: full
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: certs/test-elastic-master01.p12
xpack.security.transport.ssl.truststore.path: certs/test-elastic-master01.p12
# http SSL/TLS
http.host: 0.0.0.0
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/http.p12
6b) Hot nodes
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
cluster.name: test-elasticsearch
node.name: test-elastic-hotdata01
network.host: 10.10.4.2
discovery.seed_hosts: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
cluster.initial_master_nodes: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
node.roles: [ data,ingest ]
node.attr.box_type: hot
xpack.watcher.enabled: true
# transport SSL/TLS
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: full
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: certs/test-elastic-hotdata01.p12
xpack.security.transport.ssl.truststore.path: certs/test-elastic-hotdata01.p12
# http SSL/TLS
http.host: 0.0.0.0
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/http.p12
6c) Warm nodes
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
cluster.name: test-elasticsearch
node.name: test-elastic-warmdata01
network.host: 10.10.4.11
discovery.seed_hosts: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
cluster.initial_master_nodes: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
node.roles: [ data,ingest ]
node.attr.box_type: warm
xpack.watcher.enabled: true
# transport SSL/TLS
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: full
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: certs/test-elastic-warmdata01.p12
xpack.security.transport.ssl.truststore.path: certs/test-elastic-warmdata01.p12
# http SSL/TLS
http.host: 0.0.0.0
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/http.p12
6d) ML nodes
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
cluster.name: test-elasticsearch
node.name: test-elastic-ml01
network.host: 10.10.4.10
discovery.seed_hosts: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
cluster.initial_master_nodes: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
node.roles: [ ml ]
xpack.watcher.enabled: true
# transport SSL/TLS
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: full
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: certs/test-elastic-ml01.p12
xpack.security.transport.ssl.truststore.path: certs/test-elastic-ml01.p12
# http SSL/TLS
http.host: 0.0.0.0
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/http.p12
7) Starting elasticsearch
Start the nodes one by one using systemctl start elasticsearch, you can monitor the logs in /var/log/elasticsearch/test-elasticsearch.log
We need to ensure we remove the paramter once cluster is formed in /etc/elasticsearch/elasticsearch.yml
cluster.initial_master_nodes: ["test-elastic-master01", "test-elastic-master02", "test-elastic-master03"]
8) resetting elastic user password
you can also do this once you start the first node,
root@test-elastic-master01:/var/log/elasticsearch# /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
This tool will reset the password of the [elastic] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y
Password for the [elastic] user successfully reset.
New value: xxxxxxxxxxxxxxxxxxxxxxxx
root@test-elastic-master01:/var/log/elasticsearch#
9) Check the status of cluster and list nodes
root@test-elastic-master01:/var/log/elasticsearch# curl -X GET "https://10.10.4.2:9200/_cluster/health?pretty" -u elastic -k
Enter host password for user 'elastic':
{
"cluster_name" : "test-elasticsearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 6,
"number_of_data_nodes" : 3,
"active_primary_shards" : 2,
"active_shards" : 4,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
root@test-elastic-master01:/var/log/elasticsearch# curl -X GET "https://10.10.4.2:9200/_cat/nodes?pretty" -u elastic -k
Enter host password for user 'elastic':
10.10.4.2 2 63 0 0.04 0.05 0.02 di - test-elastic-hotdata01
10.10.4.3 2 63 0 0.00 0.06 0.06 di - test-elastic-hotdata02
10.10.4.8 7 97 1 0.00 0.10 0.09 m - test-elastic-master03
10.10.4.7 11 96 2 0.00 0.03 0.01 m * test-elastic-master02
10.10.4.6 10 97 2 0.00 0.04 0.02 m - test-elastic-master01
10.10.4.4 2 62 0 0.00 0.06 0.05 di - test-elastic-hotdata03
root@test-elastic-master01:/var/log/elasticsearch#
10) Install and Configure Kibana
10a) Installing kibana
root@test-elastic-kibana01:~# wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
root@test-elastic-kibana01:~# apt-get install apt-transport-https
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
apt-transport-https
1 upgraded, 0 newly installed, 0 to remove and 118 not upgraded.
Need to get 1,704 B of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu focal-updates/universe amd64 apt-transport-https all 2.0.9 [1,704 B]
Fetched 1,704 B in 1s (3,407 B/s)
(Reading database ... 111616 files and directories currently installed.)
Preparing to unpack .../apt-transport-https_2.0.9_all.deb ...
Unpacking apt-transport-https (2.0.9) over (2.0.8) ...
Setting up apt-transport-https (2.0.9) ...
root@test-elastic-kibana01:~#
root@test-elastic-kibana01:~# echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main
root@test-elastic-kibana01:~#
root@test-elastic-kibana01:~# sudo apt-get update && sudo apt-get install kibana
0% [Working]
Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease
Get:2 https://artifacts.elastic.co/packages/8.x/apt stable InRelease [10.4 kB]
Hit:3 http://us.archive.ubuntu.com/ubuntu focal InRelease
Hit:4 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease
Get:5 https://artifacts.elastic.co/packages/8.x/apt stable/main amd64 Packages [34.0 kB]
Hit:6 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease
Get:7 https://artifacts.elastic.co/packages/8.x/apt stable/main i386 Packages [3,556 B]
Fetched 48.0 kB in 1s (33.1 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
kibana
0 upgraded, 1 newly installed, 0 to remove and 118 not upgraded.
Need to get 285 MB of archives.
After this operation, 680 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/8.x/apt stable/main amd64 kibana amd64 8.4.3 [285 MB]
Fetched 285 MB in 3s (83.2 MB/s)
Selecting previously unselected package kibana.
(Reading database ... 111616 files and directories currently installed.)
Preparing to unpack .../kibana_8.4.3_amd64.deb ...
Unpacking kibana (8.4.3) ...
Setting up kibana (8.4.3) ...
Creating kibana group... OK
Creating kibana user... OK
Created Kibana keystore in /etc/kibana/kibana.keystore
root@test-elastic-kibana01:~#
10b) Copy the ca certificate to kibana server
Copy the ca certificate that was generated from the step 3c kibana/elasticsearch-ca.pem to /etc/kibana/elasticsearch-ca.pem
10c) Reset kibana_system password
To do below, login into one of the elasticsearch node which is added to http certificate.
root@test-elastic-master01:/var/log/elasticsearch# /usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system
This tool will reset the password of the [kibana_system] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y
Password for the [kibana_system] user successfully reset.
New value: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
root@test-elastic-master01:/var/log/elasticsearch#
10d) Configuring kibana
set below parameters in /etc/kibana/kibana.yml, we are pointing to hot data nodes below
elasticsearch.hosts: ["https://10.10.4.2:9200","https://10.10.4.3:9200","https://10.10.4.4:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "xxxxxxxxxxxxxxxxxxxxxxxx"
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/elasticsearch-ca.pem" ]
10f) Start kibana and enable the service
systemctl start kibana
systemctl enable kibana
Access kibana using elastic user using url http://kibana-hostname:5601
Top comments (0)