elk安装配置

ELK介绍

官网https://www.elastic.co/cn/

中文指南https://www.gitbook.com/book/chenryn/elk-stack-guide-cn/details

ELK Stack （5.0版本之后） Elastic Stack == （ELK Stack + Beats）

ELK Stack包含：ElasticSearch、Logstash、Kibana

ElasticSearch是一个搜索引擎，用来搜索、分析、存储日志。它是分布式的，也就是说可以横向扩容，可以自动发现，索引自动分片，总之很强大。文档https://www.elastic.co/guide/cn/elasticsearch/guide/current/index.html

Logstash用来采集日志，把日志解析为json格式交给ElasticSearch。

Kibana是一个数据可视化组件，把处理后的结果通过web界面展示

Beats在这里是一个轻量级日志采集器，其实Beats家族有5个成员

早期的ELK架构中使用Logstash收集、解析日志，但是Logstash对内存、cpu、io等资源消耗比较高。相比 Logstash，Beats所占系统的CPU和内存几乎可以忽略不计

x-pack对Elastic Stack提供了安全、警报、监控、报表、图表于一身的扩展包，是收费的

ELK架构

ELK安装 – 准备工作

准备3台机器130,132,133

角色划分：

3台全部安装elasticsearch(后续简称es) ，1主节点130,2数据节点132,133

es主130上安装kibana

1台es数据节点132上安装logstash 3台机器全部安装jdk8（openjdk即可）

yum install -y java-1.8.0-openjdk

ELK安装 – 安装es

官方文档https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html

以下操作3台机器上都要执行

rpm --importhttps://artifacts.elastic.co/GPG-KEY-elasticsearch

vim /etc/yum.repos.d/elastic.repo //加入如下内容

[elasticsearch-6.x]

name=Elasticsearch repository for 6.x packages

baseurl=https://artifacts.elastic.co/packages/6.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

yum install -y elasticsearch//也可以直接下载rpm文件，然后安装

wgethttps://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.0.0.rpm

rpm -ivh elasticsearch-6.0.0.rpm

ELK安装 – 配置es

elasticsearch配置文件/etc/elasticsearch和/etc/sysconfig/elasticsearch

参考https://www.elastic.co/guide/en/elasticsearch/reference/6.0/rpm.html

在130上编辑配置文件vim /etc/elasticsearch/elasticsearch.yml//增加或更改

cluster.name: aminglinux

node.master: true//意思是该节点为主节点

node.data: false

network.host: 0.0.0.0

discovery.zen.ping.unicast.hosts: ["192.168.133.130", "192.168.133.132", "192.168.133.133"]

配置实例：

[root@weifeng678 ~]# cat /etc/elasticsearch/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================

# NOTE: Elasticsearch comes with reasonable defaults for most settings.

#Before you set out to tweak and tune the configuration, make sure you

#understand what are you trying to accomplish and the consequences.

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

# Please consult the documentation for further information on configuration options:

#https://www.elastic.co/guide/en/elasticsearch/reference/index.html

# ---------------------------------- Cluster -----------------------------------

# Use a descriptive name for your cluster:

cluster.name: ywkd123

# ------------------------------------ Node ------------------------------------

# Use a descriptive name for the node:

node.name: ywkd01

# Add custom attributes to the node:

#node.attr.rack: r1

node.master: true

node.data: false

# ----------------------------------- Paths ------------------------------------

# Path to directory where to store the data (separate multiple locations by comma):

path.data: /var/lib/elasticsearch

# Path to log files:

path.logs: /var/log/elasticsearch

# ----------------------------------- Memory -----------------------------------

# Lock the memory on startup:

#bootstrap.memory_lock: true

# Make sure that the heap size is set to about half the memory available

# on the system and that the owner of the process is allowed to use this

# limit.

# Elasticsearch performs poorly when the system is swapping the memory.

# ---------------------------------- Network -----------------------------------

# Set the bind address to a specific IP (IPv4 or IPv6):

network.host: 0.0.0.0

# Set a custom port for HTTP:

http.port: 9200

# For more information, consult the network module documentation.

# --------------------------------- Discovery ----------------------------------

# Pass an initial list of hosts to perform discovery when new node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

discovery.zen.ping.unicast.hosts: ["192.168.4.81", "192.168.4.82","192.168.4.83"]

# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):

#discovery.zen.minimum_master_nodes:

# For more information, consult the zen discovery module documentation.

# ---------------------------------- Gateway -----------------------------------

# Block initial recovery after a full cluster restart until N nodes are started:

#gateway.recover_after_nodes: 3

# For more information, consult the gateway module documentation.

# ---------------------------------- Various -----------------------------------

# Require explicit names when deleting indices:

#action.destructive_requires_name: true

在132和133上同样编辑配置文件vim /etc/elasticsearch/elasticsearch.yml//增加或更改

cluster.name: aminglinux

node.master: false

node.data: true

network.host: 0.0.0.0

discovery.zen.ping.unicast.hosts: ["192.168.133.130", "192.168.133.132", "192.168.133.133"]

配置实例：

[root@localhost ~]# cat /etc/elasticsearch/elasticsearch.yml