Installation of Elastic-search on ubuntu

How To Install Java with Apt-Get on Ubuntu 16.04

1. Installing Oracle Java

- Go to root shell
sudo -i

- Run these commands in this order (https://www.digitalocean.com/community/tutorials/how-to-install-java-on-ubuntu-with-apt-get)

1. apt-get install python-software-properties
2. add-apt-repository ppa:webupd8team/java
3. apt-get update
4. apt-get install oracle-java8-installer
5. vim /etc/environment
6. Add this line - JAVA_HOME="/usr/lib/jvm/java-8-oracle/"
7. source /etc/environment
8. echo $JAVA_HOME

2. Install Elasticsearch with Debian package: 

Installing Elastic Search (https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html)

1. Download and install the Public Signing Key:
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

2. Install the apt-transport-https package on Debian
sudo apt-get install apt-transport-https

3. Save the repository definition to /etc/apt/sources.list.d/elasticsearch-{branch}.list:
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

4. Run apt-get update and the repository is ready for use. You can install it with:
sudo apt-get update && sudo apt-get install elasticsearch


3. Update the configuration as per requirements.
vim /etc/elasticsearch/elasticsearch.yml
- Change cluster name
- Change node name
- Change node type (master/data)
- Change Conf path
- Change Data path
- Change Log path
- Enable bootstrap.mlockall to true (Use with Caution. Elastic 5.1.1 this causes the service to crash)
- Change  network.host: 0.0.0.0 to allow binding from all IPs and DNS.
- Change Unicast Host IPs

3.1 Changes for ElasticSearch 5.X
Do these changes when installing Elastic 5.X with Memory Lock
https://www.elastic.co/guide/en/elasticsearch/reference/master/setting-system-settings.html#systemd

The systemd service file (/usr/lib/systemd/system/elasticsearch.service) contains the limits that are applied by default.

To override these, add a file called /etc/systemd/system/elasticsearch.service.d/elasticsearch.conf and specify any changes in that file, such as:

[Service]
LimitMEMLOCK=infinity

- Create a directory /etc/systemd/system/elasticsearch.service.d
- Add File elasticsearch.conf
- Add the line
[Service]
LimitMEMLOCK=infinity

4. Update JVM Options to increase Heap Size
1. vim /etc/elasticsearch/jvm.options
2. Change -Xms2g to -Xms15g
3. Change -Xmx2g to -Xmx15g

5. Increase number of file descriptors and Threads
1. Open /etc/security/limits.conf and add these lines

*                soft    nofile          100000
*                hard    nofile          200000
root             soft    nofile          100000
root             hard    nofile          200000
*                soft    nproc           2048
*                hard    nproc           2048
root             soft    nproc           2048
root             hard    nproc           2048

2. Add in permissions file /etc/pam.d/common-session

session required        pam_limits.so

6. Update the auto start script
vim /etc/init.d/elasticsearch
- Change ES_USER=maelasticuserdev (or any other admin user)
- Change ES_GROUP=adm (ensure user in admin group)
- Allocate Heap Size (ES_HEAP_SIZE)
- Change MAX_LOCKED_MEMORY=unlimited (check if mlockall is true)
- Change LOG_DIR
- Change DATA_DIR
- Change WORK_DIR
- Add these lines after startup to source the global variables
#Source the environment variables
#Post restart the variables are not initialized
echo "Sourcing Environemnt variables for $ES_MIN_MEM and $ES_MAX_MEM"
. /etc/environment
echo $ES_MIN_MEM
echo $ES_MAX_MEM

- comment the line
start-stop-daemon --start -b --user "$ES_USER" -c "$ES_USER" --pidfile "$PID_FILE" --exec $DAEMON -- $DAEMON_OPTS
- also comment the retry section
- Add this startup script
$DAEMON $DAEMON_OPTS
echo "Sleeping for 10 seconds till process starts"
sleep 10
- comment stop code
#               start-stop-daemon --stop --pidfile "$PID_FILE" \
#                       --user "$ES_USER" \
#                       --quiet \
#                       --retry forever/TERM/20 > /dev/null
#               if [ $? -eq 1 ]; then
#                       log_progress_msg "$DESC is not running but pid file exists, cleaning up"
#               elif [ $? -eq 3 ]; then
#                       PID="`cat $PID_FILE`"
#                       log_failure_msg "Failed to stop $DESC (pid $PID)"
#                       exit 1
#               fi
- Add this stop script
 pid=`cat $PID_FILE`
             echo "Process Id: $pid"
             kill $pid

7. Configure Elasticsearch to automatically start during bootup.

For systemd

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

Starting and Stopping
sudo systemctl start elasticsearch.service
sudo systemctl stop elasticsearch.service

For sysV init

sudo update-rc.d elasticsearch defaults 95 10

sudo -i service elasticsearch start
sudo -i service elasticsearch stop

8. For Replicated / VM Snapshot Machines

Make sure to delete all data folders for the new created VM from a snapshot.
Elastic stores the node id in the data folder which is not cleared unless the data files are deleted.

Author: Mohammed Adil