Asyncio标准输出 - 失败
问题描述:
我正试图抓住使用asyncio来加速调用外部工具来分析多个音频文件的过程。我正在研究windows,python 3.6(anaconda安装),我在这里遇到的问题是调用似乎没有等待,或者结果从来没有通过stdout接收。Asyncio标准输出 - 失败
任何想法?
import os
import asyncio
external_tool = r"C:\path\to\external\tool.exe"
def filepaths_generator(root):
'''
Simple generator to yield filepaths
'''
for path, dirs, files in os.walk(root):
for f in files:
yield os.path.join(path,f)
async def async_subprocess_command(*args):
'''
A function to run the asyncio subprocess call
'''
# Create subprocess
process = asyncio.create_subprocess_exec(
*args,
# stdout must a pipe to be accessible as process.stdout
stdout=asyncio.subprocess.PIPE)
# Wait for the subprocess to finish
stdout, stderr = await process.communicate()
# Return stdout
return stdout.decode().strip()
async def add_external_tool_data_to_dict(filepath):
external_tool_data = await async_subprocess_command(external_tool,filepath)
metadata_dict[filepath] = external_tool_data
return
metadata_dict = {}
loop = asyncio.get_event_loop()
#loop = asyncio.ProactorEventLoop() - Tried this, it doesn't help!
filepaths = filepaths_generator(r"C:\root\path")
tasks = []
for filepath in filepaths:
#Create tasks in for the loop and add to a list
task = loop.create_task(add_external_tool_data_to_dict(filepath))
tasks.append(task)
#Run the tasks
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
print(metadata_dict)
答
对于初学者有this:
On Windows, the default event loop is SelectorEventLoop which does not support subprocesses. ProactorEventLoop should be used instead
示例使用它在Windows上:
import asyncio, sys
if sys.platform == 'win32':
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
我会切换到该替换事件循环开始。
答
按Gerrat的评论,该解决方案是使用ProactorEventLoop和其后添加一个额外的行:
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
如果我使用循环= asyncio.ProactorEventLoop()我得到的错误: – user3535074
文件“C: \用户\丹尼尔\ Anaconda3 \ LIB \ ASYNCIO \ base_events.py”,线路340,在_make_subprocess_transport 提高NotImplementedError NotImplementedError – user3535074
后你试过了'循环= asyncio.ProactorEventLoop()',并跟进:'asyncio.set_event_loop (循环)'? – Gerrat