[问题处理]filebeat收集json格式的nginx日志
filebeat收集json格式的nginx日志
由于画蛇添足 用了下面的配置导致nginx的json日志一直显示在message里
log_format access_json '{ "@timestamp": "$time_local", ' '"remote_addr": "$remote_addr", ' '"referer": "$http_referer", ' '"request": "$request", ' '"status": $status, ' '"bytes": $body_bytes_sent, ' '"agent": "$http_user_agent", ' '"x_forwarded": "$http_x_forwarded_for", ' '"up_addr": "$upstream_addr",' '"up_host": "$upstream_http_host",' '"up_resp_time": "$upstream_response_time",' '"request_time": "$request_time"' '"remote_user ": "$remote_user "' '"http_host ": "$http_host "' '"upstream_status": "$upstream_status"' '"http_referer": "$http_referer"' '"ssl_protocol": "$ssl_protocol"' '"ssl_cipher ": "$ssl_cipher "' ' }'; |
下面是nginx输出
导致filebeat报错
修改nginx.conf中https模块
log_format access_json '{ "@timestamp": "$time_iso8601", ' '"time": "$time_iso8601", ' '"remote_addr": "$remote_addr", ' '"remote_user": "$remote_user", ' '"body_bytes_sent": "$body_bytes_sent", ' '"request_time": "$request_time", ' '"status": "$status", ' '"host": "$host", ' '"request": "$request", ' '"request_method": "$request_method", ' '"uri": "$uri", ' '"http_referrer": "$http_referer", ' '"body_bytes_sent":"$body_bytes_sent", ' '"http_x_forwarded_for": "$http_x_forwarded_for", ' '"http_user_agent": "$http_user_agent" ' '}';
|
下面是nginx输出
然后修改filebeat.yml
加入 json.keys_under_root: true
json.overwrite_keys: true
json.keys_under_root: 默认这个值是FALSE的,也就是我们的json日志解析后会被放在json键上。设为TRUE,所有的keys就会被放到根节点
json.overwrite_keys: 是否要覆盖原有的key,这是关键配置,将keys_under_root设为TRUE后,再将overwrite_keys也设为TRUE,就能把filebeat默认的key值给覆盖了
|
然后reload nginx 重启filebeat 之后的结果