您的位置: 首页  >  技术问答  >  将JedisPool与Tomcat配合使用，资源不会返回池

将JedisPool与Tomcat配合使用，资源不会返回池

分类: 技术问答 • 2022-04-20 20:16:42
问题描述：

看看redis client list的输出，我发现目前有600个活动客户端，并且它不断增长。下面是输出的一个片段：将JedisPool与Tomcat配合使用，资源不会返回池

id=285316 addr=x.x.x.x:55699 fd=14131 name= age=53055 idle=53029 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=sismember id=285317 addr=x.x.x.x:55700 fd=14132 name= age=53055 idle=53050 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=sismember

这里是我的代码：

Listener.java：

import com.sun.jersey.api.model.AbstractResourceModelContext; 
import com.sun.jersey.api.model.AbstractResourceModelListener; 

import javax.ws.rs.ext.Provider; 

@Provider 
public class Listener implements AbstractResourceModelListener { 

    @Override 
    public void onLoaded(AbstractResourceModelContext modelContext) { 
     RedisManager.getInstance().connect(); 
    } 

}

RedisManager.java：

import redis.clients.jedis.Jedis; 
import redis.clients.jedis.JedisPool; 
import redis.clients.jedis.JedisPoolConfig; 

public class RedisManager { 
    private static final RedisManager instance = new RedisManager(); 
    private static JedisPool pool; 

    private RedisManager() { 
    } 

    public final static RedisManager getInstance() { 
     return instance; 
    } 

    public void connect() { 
     JedisPoolConfig poolConfig = new JedisPoolConfig(); 
     poolConfig.setMaxTotal(5000); 
     poolConfig.setTestOnBorrow(true); 
     poolConfig.setTestOnReturn(true); 
     poolConfig.setMaxIdle(50); 
     poolConfig.setMinIdle(1); 
     poolConfig.setTestWhileIdle(true); 
     poolConfig.setNumTestsPerEvictionRun(10); 
     poolConfig.setTimeBetweenEvictionRunsMillis(60000); 
     pool = new JedisPool(poolConfig, "redis_hostname"); 
    } 

    public void release() { 
     pool.destroy(); 
    } 

    public Jedis getJedis() { 
     return pool.getResource(); 
    } 

    public void returnJedis(Jedis jedis) { 
     pool.returnResourceObject(jedis); 
    } 
}

APIServlet.java：

@Path("/") 
public class APIService { 

    @GET 
    @Path("/lookup") 
    @Produces(MediaType.APPLICATION_JSON) 
    public Response getMsg(@QueryParam("email") String email, 
          @QueryParam("pretty") String pretty 
    ) throws JSONException { 
     Jedis jedis = RedisManager.getInstance().getJedis(); 
     if (jedis.sismember("inprocess", email)) { 
      RedisManager.getInstance().returnJedis(jedis); 
      return Response.status(202).entity("{\"status\":202, " + 
        "\"processing\":{\"type\":\"Lookup performed\", " + 
        "\"message\":\"We're performing analysis on this " + 
        "record. Result should be ready in a few minutes" + 
        ".\"}}").build(); 
     } 

     Person person = new Person(); 
     person.lookup(person); 
     ObjectMapper mapper = new ObjectMapper(); 
     String jsonString = mapper.writeValueAsString(person); 
     JSONObject jsonObj = new JSONObject(jsonString);   
     jsonObj.remove("objectID"); 
     jsonObj.remove("data_quality"); 
     jsonObj.put("status", 200); 

     RedisManager.getInstance().returnJedis(jedis); 

     if (!jsonObj.isNull("name") && !jsonObj.get("name").equals("")) { 
      if (hasPretty) { 
       return Response.status(200).entity(jsonObj.toString(4)) 
        .build(); 
      } 
      return Response.status(200).entity(jsonObj.toString()).build(); 
     } 

     return Response.status(404).entity("{\"status\":404, " + 
       "\"error\":{\"type\":\"Data Not Found.\", " + 
       "\"message\":\"We were not able to find data " + 
       "on this email.\"}}").build(); 
    } 
}

Maven依赖：

<dependency> 
     <groupId>com.sun.jersey</groupId> 
     <artifactId>jersey-server</artifactId> 
     <version>1.8</version> 
    </dependency> 
    <dependency> 
     <groupId>com.sun.jersey</groupId> 
     <artifactId>jersey-json</artifactId> 
     <version>1.8</version> 
    </dependency> 
    <dependency> 
     <groupId>redis.clients</groupId> 
     <artifactId>jedis</artifactId> 
     <version>2.7.2</version> 
     <type>jar</type> 
     <scope>compile</scope> 
    </dependency> 
    <dependency> 
     <groupId>commons-validator</groupId> 
     <artifactId>commons-validator</artifactId> 
     <version>1.2.0</version> 
    </dependency>

监听器创建RedisManager的一个实例，在整个应用程序中使用 - 这应该只发生一次，在启动（注意：我不知道如何在关机时调用destroy，这很好理解）。在整个程序中，JedisPool的这个实例在Jersey路由中使用，如APIServlet.java中所示。在路由中，我得到一个JedisPool资源，然后在返回任何路由的任何部分之前，我返回资源。

发生什么事是资源似乎没有被返回（或我对池的理解是错误的）。一段时间后，与我的Redis实例的连接增长到maxTotal为5,000，然后我开始发生错误“无法从池中获取资源”，并且Tomcat死亡。

有几件事情我已经注意到：

  1. 似乎有大量的是坚持各地建立HTTPS连接（在此一定不要100％，但它似乎是这样）。

  2. 所有闲置的Redis客户端（几乎所有的）都有一个cmd的sismember。

注：我不包括全APIService.java代码，因为我真的不能这样做。我包含的片段确实给出了代码的总体要点。我返回整个APIService.java代码（返回404，返回429等），并且在每次返回之前，我确保将资源返回到池中。

最后，这里的堆栈跟踪：

10-Feb-2016 08:04:23.161 SEVERE [http-nio-443-exec-14] com.sun.jersey.spi.container.ContainerResponse.mapMappableContainerException The RuntimeException could not be mapped to a response, re-throwing to the HTTP container 
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool 
     at redis.clients.util.Pool.getResource(Pool.java:50) 
     at redis.clients.jedis.JedisPool.getResource(JedisPool.java:86) 
     at co.talentiq.api.APIService.getMsg(APIService.java:63) 
     at sun.reflect.GeneratedMethodAccessor52.invoke(Unknown Source) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:498) 
     at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) 
     at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) 
     at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) 
     at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) 
     at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) 
     at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) 
     at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) 
     at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) 
     at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) 
     at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) 
     at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) 
     at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) 
     at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) 
     at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) 
     at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699) 
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) 
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) 
     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) 
     at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) 
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) 
     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) 
     at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) 
     at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) 
     at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) 
     at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) 
     at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) 
     at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616) 
     at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) 
     at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521) 
     at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096) 
     at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674) 
     at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500) 
     at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
     at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) 
     at java.lang.Thread.run(Thread.java:745)

答

第一：如果你已经有池初始化不创建一个新的：

public class RedisManager { 
... 
public void connect() { 
    if(pool != null) { 
     System.out.println("Already exists"); 
     return; 
    } 
    JedisPoolConfig poolConfig = new JedisPoolConfig(); 
    ...

二...你有你的日志异常来自getMsg方法？

public Response getMsg(@QueryParam("email") String email, 
         @QueryParam("pretty") String pretty

你应该把所有的工作都包含在try-catch-finally中并且总是返回finally块中的资源。注意：确保不要返回资源（在这种情况下是jedis）来池两次。

Jedis jedis; 
try { 
    jedis = RedisManager.getInstance().getJedis(); 
    ... 
} finally { 
    if (jedis != null) { 
     RedisManager.getInstance().returnJedis(jedis); 
     jedis = null; 
    } 
}

顺便说一句：你可以创建一个小AutoCloseable来包裹你的jedis获取/返回代码和使用Java尝试与资源 - https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html

代码段使用try-与资源

public void release() { 
    pool.destroy(); 
} 
public static class JedisWrapper implements AutoCloseable { 
    private final JedisPoolConfig pool; 
    private final Jedis jedis; 
    public JedisWrapper(JedisPoolConfig pool, Jedis jedis) { 
     this.pool = pool; 
     this.jedis = jedis; 
    } 
    public Jedis get() { 
     return jedis; 
    } 
    @Override 
    public void close() { 
     pool.returnResourceObject(jedis); 
    } 
} 
public JedisWrapper getJedis() { 
    return new JedisWrapper(pool, pool.getResource()); 
} 
// you can delete this method 
public void returnJedis(Jedis jedis) { 
    pool.returnResourceObject(jedis); 
}

后来在使用的地方

public Response getMsg(@QueryParam("email") String email, 
         @QueryParam("pretty") String pretty 
) throws JSONException { 
    try(JedisWrapper jw = ...) { 
     Jedis jedis = jw.get(); 
     ... 
    }
+0

是否有区别pool.return ResourceOjbect（）和pool.close（）？从我一直在读的内容来看，pool.close（）是做事情的正确方法，但是从我对池的理解中，似乎你只想返回资源，以便它可以被重用。我目前正在实施您的建议，并将很快进行测试。感谢您的反馈！ –

+0

@FranzKafka pool.returnObject - 将返回单个采集对象，pool.close将关闭该池以及池中注册的所有对象。我更新了一个示例如何将代码转换为autoclose方法的答案。使用记事本在3分钟内创建示例。我认为你可以更好地来。 –

相关推荐