python多处理酸洗/管理器/ misc错误(来自PMOTW)
我在通过Windows在Eclipse上运行以下代码时遇到了一些麻烦。该代码是从Doug Hellman:python多处理酸洗/管理器/ misc错误(来自PMOTW)
import random
import multiprocessing
import time
class ActivePool:
def __init__(self):
super(ActivePool, self).__init__()
self.mgr = multiprocessing.Manager()
self.active = self.mgr.list()
self.lock = multiprocessing.Lock()
def makeActive(self, name):
with self.lock:
self.active.append(name)
def makeInactive(self, name):
with self.lock:
self.active.remove(name)
def __str__(self):
with self.lock:
return str(self.active)
def worker(s, pool):
name = multiprocessing.current_process().name
with s:
pool.makeActive(name)
print('Activating {} now running {}'.format(
name, pool))
time.sleep(random.random())
pool.makeInactive(name)
if __name__ == '__main__':
pool = ActivePool()
s = multiprocessing.Semaphore(3)
jobs = [
multiprocessing.Process(
target=worker,
name=str(i),
args=(s, pool),
)
for i in range(10)
]
for j in jobs:
j.start()
for j in jobs:
j.join()
print('Now running: %s' % str(pool))
我碰到下面的错误,我以为是由于某些酸洗问题与pool
作为参数传递给Process
。
Traceback (most recent call last):
File "E:\Eclipse_Workspace\CodeExamples\FromCodes\CodeTest.py", line 50, in <module>
j.start()
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
reduction.dump(process_obj, to_child)
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\connection.py", line 939, in reduce_pipe_connection
dh = reduction.DupHandle(conn.fileno(), access)
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\connection.py", line 170, in fileno
self._check_closed()
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\connection.py", line 136, in _check_closed
raise OSError("handle is closed")
OSError: handle is closed
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\spawn.py", line 99, in spawn_main
new_handle = reduction.steal_handle(parent_pid, pipe_handle)
File "C:\Users\Bob\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\reduction.py", line 87, in steal_handle
_winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied
类似的问题的answer似乎表明,我初始化pool
与顶层的函数调用,但我不知道如何适用于这个例子。我是否在worker
中初始化ActivePool
?这似乎打败了赫尔曼的榜样精神。
另一个answer建议我用__getstate__
,__setstate__
,除去unpickleable对象和取储存时重建他们,但我不知道一个很好的方式与代理对象像经理要做到这一点,我居然不知道是什么不可取消的对象是。
有没有什么办法可以让这个例子在最小的变化下工作?我真的很想了解底下发生了什么。谢谢!
编辑 - 问题解决了:
酸洗问题是在事后很明显。 ActivePool的__init__
包含一个Manager()对象,看起来不可取。该代码运行正常按照赫尔曼的例子,如果我们去掉self.mgr
,并在一行初始化列表ProxyObject:
def __init__(self):
super(ActivePool, self).__init__()
self.active = multiprocessing.Manager().list()
self.lock = multiprocessing.Lock()
Comment: The 'join()' was in the Hellman example, but I forgot to add it into the code snippet. Any other ideas?
我运行Linux和它按预期工作时,Windows表现不同的阅读understanding-multiprocessing-shared-memory-management-locks-and-queues-in-pyt
要确定哪个参数args=(s, pool)
引发错误,请删除一个并将其用作global
。
变化:
def worker(s):
...
args=(s,),
Note: There is no need to enclose a
multiprocessing.Manager().list()
with aLock()
.
This is not the culprit of your error.
Question: Is there any way I can make this example work with minimal changes?
你__main__
过程终止,因此,所有的启动的进程在执行不可预测的位置死亡。添加简单的一个.join()
末让__main__
等到所有进程完成:
for j in jobs:
j.join()
print('EXIT __main__')
测试使用Python 3.4.2
我评论了所有的锁调用和得到同样的错误。 'join()'在Hellman示例中,但我忘了将它添加到代码片段中。任何其他想法? – Zhulu
在我解决问题之前,我创建了一个全局信号量和一个全局池句柄,然后使用'args(s,)'''args(pool,)''''args(s,pool)'运行代码。所有这三个不幸都没有工作。第二和第三种方法导致同样的错误和第一种方法导致'AttributeError的:“NoneType”对象没有属性“makeActive''。我所做的全局和初始化它们喜欢:'池= None'和 '高清initialize_pool(): 全局池\ 池= ActivePool()'。 – Zhulu