ELK(elasticsearch stack)日志搭建
安装cenos
ifconfig
vi/etc/sysconfig/network-scripts/ifcfg-ens33
onboot=yes.
service network restart
ifconfig
安装全局jdk
1、 tar -zxvf /usr/local/jdk-8u131-linux-x64.tar.gz
2、vi /etc/profile
3、
JAVA_HOME=/usr/local/jdk1.8.0_131
JRE_HOME=/usr/local/jdk1.8.0_131/jre
CLASSPATH=.:$JAVA_HOME/lib:/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$PATH:$JAVA_HOME/bin
export JAVA_HOME
export JRE_HOME
ulimit -u 4096
4、source /etc/profile
创建用户关闭防火墙
[root@localhost local]#
vi /etc/security/limits.conf
添加以下内容
* soft nproc 65536
* hard nproc 65536
* soft nofile 65536
* hard nofile 65536
创建运行ELK的用户(至于为啥要建立elk用户,好像看过一篇文章说root用户不能运行elasticsearch)
[[email protected] local]# groupadd elk
[[email protected] local]# useradd -g elk elk
# passwd elk
密码:elk
创建ELK运行目录
[[email protected] local]# mkdir /elk
[[email protected] local]# chown -R elk:elk /elk
关闭防火墙:
[[email protected] ~]# iptables -F
以上全部是root用户完成
Elasticsearch配置与安装(搜索,提供分布式全文搜索引擎)
案例:https://www.cnblogs.com/leixingzhi7/p/6844977.html
登录elk用户
复制elk三个tar包到elk目录下
解压:tar –zxvf
# vielasticsearch-5.4.0/config/elasticsearch.yml
修改如下内容:
cd elasticsearch-5.4.0
./bin/elasticsearch
结果报错,解决办法:
1、切换到root用户修改配置sysctl.conf
vi /etc/sysctl.conf
添加下面配置:
vm.max_map_count=655360
并执行命令:
sysctl-p
再重启:./bin/elasticsearch
netstat –ant 查看是否成功
Logstach配置与安装(日志收集,管理,存储)
vi ./conf/logstash.conf
编写:
input {
file {
path => "/home/elk/data/logs/message_center.log"//日志路径。可以用*代替
type => "log"//type类型,下面ouput筛选使用,多个file时有用
start_position => "beginning"//从开头开始
codec => multiline {//曾泽表达式过滤日志条数
pattern => "^\["//正则表达式
negate => true
what => "previous"
}
}
}
output {
if [type] == "log" {
elasticsearch {
hosts => ["192.168.48.138:9200"]//elasticsearch安装路径
index => "log-%{+YYYY.MM.dd}"//搜索名称命名方式
}
}
}
Copy下面代码:
input {
file {
path => "/home/elk/data/logs/message_center.log"
type => "log"
start_position => "beginning"
codec => multiline {
pattern => "^\["
negate => true
what => "previous"
}
}
}
output {
elasticsearch {
hosts => ["192.168.48.138:9200"]
index => "log-%{+YYYY.MM.dd}"
}
}
启动logstash:
./bin/logstash -f ./config/logstash.conf
1、其中要修改配置文件:logstash停止ctrl+c 不能停止
查看logstash状态:
systemctl status logstash.service
如果未停止:
systemctl stop logstash.service或service logstash stop
2、报错
Logstash could not be started becausethere is already another instance using the configured data directory. Ifyou wish to run multiple instances, you must change the "path.data"setting
解决办法:启动时
./bin/logstash -f ./config/logstash.conf --path.data=/root/
kibana配置与安装(日志的过滤web 展示)
cd kibana-5.4.0-linux-x86_64
vi config/kibana.yml
启动:
./bin/kibana
表示访问成功!