Linux环境下将log4j的日志存储到mongoDB

转自：http://my.oschina.net/chinaearl/blog/111552

单位有很多java项目，一般上生产的项目日志都是通过log4j打印在本地，平常开发同学要查看日志还要给他们开个专门的FTP，不利于集中维护管理，近日因工作需要，在网上看到相关技术资料，可把log4j日志直接存放到mongoDB，在mongoDB的可以更好的查询日志。非常利于集中管理和维护。以下是自己和开发同学配置部署的过程，可谓是一波三折：

Log4mongo for Java官方介绍：

MongoDbAppender - stores the LogEvent in a BSONified form
ExtendedMongoDbAppender - Extends MongoDbAppender by allowing you to add top level elements
MongoDbPatternLayoutAppender - supports logging data in a custom format

1.首先下载并安装mongoDB，并运行，下载安装请参考我的上篇博文
版本要求（MongoDB Server v2.0+ (tested with 2.2.2)）

2.下载Mongo Java driver jar包

3.下载 Log4J jar包注意版本，一定要1.2.16及以上。

4.下载 log4mongo-java jar包

5.请将这三个jar文件放置在你的项目classpath中。

6.在你的classpath中创建一个名为log4j.properties的文件

7.在log4j.properties文件尾添加以下几行

1
2
3
4

log4j.rootLogger=INFO, MongoDB                              
log4j.appender.MongoDB=org.log4mongo.MongoDbAppender        
log4j.appender.MongoDB.databaseName=appname                 
log4j.appender.MongoDB.collectionName=log                   

如果你的mongoDB不是在本地，那添加mongoDB数据库的IP地址和端口

1
2


log4j.appender.MongoDB.hostname=172.17.0.9  
log4j.appender.MongoDB.port=27017

创建一个Java类，如：

1
2
3
4
5
6
7
8


import org.apache.log4j.Logger;
 
public class LogTest {

    public static void main(String[] args) {

        Logger logger = Logger.getLogger(LogTest.class);

        logger.error("Don't panic");

    }
}

编译和运行这个类

碰到问题，mongoDB里的日志数据始终不见出来。后来发现问题，原因是项目要部署到tomcat上才可能生效。

以下是开发同学整理的部署情况

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

1、建立一个web项目，项目名为logmongo，在项目当中引入jar包，见步骤2~4（指你写的）描述，共3个jar包。
2、log4j.properties内容：
log4j.rootLogger=debug, stdout, R, MongoDB
 
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

log4j.appender.stdout.layout.ConversionPattern=%-d{MM-dd HH:mm:ss.SSS} %-5p - [%c{1}.%t] %m%n
 
log4j.appender.R=org.apache.log4j.RollingFileAppender

log4j.appender.R.File=logs/client.log
log4j.appender.R.MaxFileSize=1000KB
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout

log4j.appender.R.layout.ConversionPattern=%-d{MM-dd HH:mm:ss.SSS} %-5p - [%c{1}.%t] %m%n
 
log4j.appender.MongoDB=org.log4mongo.MongoDbAppender
log4j.appender.MongoDB.databaseName=appname
log4j.appender.MongoDB.collectionName=log

log4j.appender.MongoDB.hostname=172.17.0.9
log4j.appender.MongoDB.port=27017
 
3、web.xml内容

<?xml version="1.0" encoding="UTF-8"?>

<web-app version="3.0" xmlns="http://java.sun.com/xml/ns/javaee"

 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

 xsi:schemaLocation="http://java.sun.com/xml/ns/javaee

 http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd">

 <display-name></display-name>

 <welcome-file-list>

 <welcome-file>index.jsp</welcome-file>

 </welcome-file-list>
 

 <context-param>

 <param-name>log4jConfigLocation</param-name>

 <param-value>classpath:/log4j.properties</param-value>

 </context-param>

  

 <servlet>

 <servlet-name>log4jServlet</servlet-name>

 <servlet-class>log4jServlet</servlet-class>

 </servlet>
 

 <servlet-mapping>

 <servlet-name>log4jServlet</servlet-name>

 <url-pattern>/log4jServlet</url-pattern>

 </servlet-mapping>

</web-app>
 
4、log4jServlet.java

import java.io.IOException;

import javax.servlet.http.HttpServlet;

import javax.servlet.http.HttpServletRequest;

import javax.servlet.http.HttpServletResponse;

import org.apache.log4j.Logger;
 
public class log4jServlet extends HttpServlet {

    private static Logger logger = Logger.getLogger(log4jServlet.class);
 

    public void doPost(HttpServletRequest request, HttpServletResponse response)

            throws IOException {

        this.doGet(request, response);

    }
 

    public void doGet(HttpServletRequest request, HttpServletResponse response)

            throws IOException {

        logger.error("test for tomcat");

    }
 
}
 

5、部署到tomcat上运行，浏览器中输入http://localhost:8080/logmongo/log4jServlet即可。

然后查看mongoDB，看是否有记录

[[email protected] bin]# ./mongo
MongoDB shell version: 2.2.2

connecting to: test
> show dbs
admin   (empty)
appname 0.203125GB

local   (empty)
> use appname
switched to db appname
> db.log.findOne()
{

    "_id" : ObjectId("513427c2abef7a8b93610a70"),

    "timestamp" : ISODate("2013-03-04T04:49:06.060Z"),

    "level" : "ERROR",

    "thread" : "http-8080-1",

    "message" : "Don't panic",

    "loggerName" : {

        "fullyQualifiedClassName" : "log4jServlet",

        "package" : [

            "log4jServlet"

        ],

        "className" : "log4jServlet"

    },

    "fileName" : "log4jServlet.java",

    "method" : "doGet",

    "lineNumber" : "24",

    "class" : {

        "fullyQualifiedClassName" : "log4jServlet",

        "package" : [

            "log4jServlet"

        ],

        "className" : "log4jServlet"

    },

    "host" : {

        "process" : "[email protected]",

        "name" : "yangyang",

        "ip" : "10.1.19.2"

    }
}

接下去，可以用PHP写个查看mongoDB数据的程序，友好的把日志通过网页形式显示出来。

MongoDB的PHP扩展安装

1.下载php扩展包：http://pecl.php.net/package/mongo

2.安装php的mongodb扩展

1
2
3
4
5
6


tar zxvf mongo-1.3.4.tgz

cd mongo-1.3.4/
#/usr/local/webserver/php/ 您的php实际安装目录  
/usr/local/webserver/php/bin/phpize

./configure --with-php-config=/usr/local/webserver/php/bin/php-config

make && make install

php读取mongodb日志数据程序如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

<?php

    error_reporting(7);

    $conn = new Mongo();

    $db = $conn->appname;

    $collection = $db->log;
?>

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>

 <head>

  <title> log4j for mongodb </title>

  <meta name="Generator" content="EditPlus">

  <meta name="Author" content="">

  <meta name="Keywords" content="">

  <meta name="Description" content="">

<style type="text/css">  

        table {  

            border: 1px solid #999999;  

            padding:0;   

            margin:0 auto;  

            border-collapse: collapse;  

        }  

           

        td,th {  

            border: 1px solid #999999;  

            background: #fff;  

            font-size:13px;  

            padding: 3px 3px 3px 8px;  

            color: #000;  

        }  
</style> 

 </head>
 

 <body>
 

  <table border="1" width="100%">
 

  <tr>

    <th rowspan="2">ID</th>

    <th rowspan="2">TIMESTAMP</th>

    <th rowspan="2">LEVEL</th>

    <th rowspan="2">THREAD</th>

    <th rowspan="2">MESSAGE</th>

    <th colspan="3">LOGGERNAME</th>

    <th rowspan="2">FILENAME</th>

    <th rowspan="2">METHOD</th>

        <th rowspan="2">lineNumber</th>

    <th colspan="3">CLASS</th>

    <th colspan="3">HOST</th>

  </tr>

  <tr>

    <td>fullyQualifiedClassName</td>

    <td>package</td>

    <td>className</td>

    <td>fullyQualifiedClassName</td>

    <td>package</td>

    <td>className</td>

    <td>process</td>

    <td>name</td>

    <td>ip</td>

  </tr>
 
<?php

    $cursor = $collection->find();

    while($cursor->hasNext()){

        $r = $cursor->getNext();
?>

  <tr>

        <td><?php echo $r[_id]?></td>

        <td><?php echo $r[timestamp]?></td>

        <td><?php echo $r[level]?></td>

        <td><?php echo $r[thread]?></td>

        <td><?php echo $r[message]?></td>

        <td><?php echo $r[loggerName][fullyQualifiedClassName]?></td>

        <td><?php echo implode('.', $r[loggerName][package]);?></td>

        <td><?php echo $r[loggerName][className]?></td>

        <td><?php echo $r[fileName]?></td>

        <td><?php echo $r[method]?></td>

        <td><?php echo $r[lineNumber]?></td>

        <td><?php echo $r['class'][fullyQualifiedClassName];?></td>

        <td><?php echo implode('.', $r['class'][package]);?></td>

        <td><?php echo $r['class'][className];?></td>

        <td><?php echo $r[host][process]?></td>

        <td><?php echo $r[host][name]?></td>

        <td><?php echo $r[host][ip]?></td>

  </tr>
<?php
}
?>

  </table>
 

 </body>
</html>

运行效果如图:

另：log4j还可以通过配置写到MySQL数据库里去。不过论现在形式而言，存到nosql里去会更优雅.

Linux环境下将log4j的日志存储到mongoDB

相关推荐