芹菜kombu失败后,self.connections.acquire

问题描述:

当我的芹菜服务运行7-10天后,我收到了这个异常从不知从哪里,这导致我的任务不被处理。重启芹菜可以解决问题。芹菜kombu失败后,self.connections.acquire

INTERNAL ERROR: RuntimeError('Acquire on closed pool',) 
Traceback (most recent call last): 
    File "/usr/lib/python2.7/dist-packages/celery/app/trace.py", line 253, in trace_task 
    I, R, state, retval = on_error(task_request, exc, uuid) 
    File "/usr/lib/python2.7/dist-packages/celery/app/trace.py", line 201, in on_error 
    R = I.handle_error_state(task, eager=eager) 
    File "/usr/lib/python2.7/dist-packages/celery/app/trace.py", line 85, in handle_error_state 
    }[self.state](task, store_errors=store_errors) 
    File "/usr/lib/python2.7/dist-packages/celery/app/trace.py", line 118, in handle_failure 
    req.id, exc, einfo.traceback, request=req, 
    File "/usr/lib/python2.7/dist-packages/celery/backends/base.py", line 121, in mark_as_failure 
    traceback=traceback, request=request) 
    File "/usr/lib/python2.7/dist-packages/celery/backends/amqp.py", line 124, in store_result 
    with self.app.amqp.producer_pool.acquire(block=True) as producer: 
    File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 868, in acquire 
    R = self.prepare(R) 
    File "/usr/lib/python2.7/dist-packages/kombu/pools.py", line 63, in prepare 
    conn = self._acquire_connection() 
    File "/usr/lib/python2.7/dist-packages/kombu/pools.py", line 38, in _acquire_connection 
    return self.connections.acquire(block=True) 
    File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 859, in acquire 
    raise RuntimeError('Acquire on closed pool') 
RuntimeError: Acquire on closed pool 

软件版本

software -> celery:3.1.20 (Cipater) kombu:3.0.35 py:2.7.6 
      billiard:3.3.0.22 py-amqp:1.4.9 
platform -> system:Linux arch:64bit, ELF imp:CPython 
loader -> celery.loaders.default.Loader 
settings -> transport:amqp results:amqp 

CELERY_ACCEPT_CONTENT: ['json', 'pickle', 'yaml'] 
CELERY_ENABLE_UTC: True 
CELERY_IGNORE_RESULT: False 
CELERY_IMPORTS: 
    ('catalogue.app.voice.cluster.deploy_cluster', 
'catalogue.app.common.install_uc', 
'hypervisor.app.deploy_esx', 
'hypervisor.app.vm_operations', 
'tools.deploy_tools') 
CELERYD_CHDIR: '/usr/local/src/imbue/application/app' 

CELERY_TASK_RESULT_EXPIRES: 18000 
CELERY_RESULT_PERSISTENT: True 
CELERY_TIMEZONE: 'US/Eastern' 
BROKER_URL: 'amqp://******:********@rabbitmq:5672//' 
CELERY_RESULT_BACKEND: 'amqp' 

唯一的解决方法是现在重新启动。 Ubuntu 14.04 2 GB RAM/2 CPU/40 GB HDD

这看起来像是bug in celery。 Asksol修复了这个few days back

您可以从源代码安装芹菜并尝试它。如果它仍然造成问题,请在github上创建新问题。