如何进行未来调用并等到Python完成？

问题描述：

我有下面的代码，我有一个用户名列表，我尝试和检查，如果用户使用net user \domain | find somegroup是在一个特定的Windows用户组。如何进行未来调用并等到Python完成？

的问题是，我可以针对每个用户名约8群组该命令，它是缓慢的。我想用期货发送这些电话，甚至分开线程（如果它使速度更快）。

我只需要在年底前等待我做别的。我如何着手在Python中做这件事？

for one_username in user_list: 
    response = requests.get(somecontent) 

    bs_parsed = BeautifulSoup(response.content, 'html.parser') 

    find_all2 = bs_parsed.find("div", {"class": "QuickLinks"}) 
    name = re.sub("\s\s+", ' ', find_all2.find("td", text="Name").find_next_sibling("td").text) 

    find_all = bs_parsed.find_all("div", {"class": "visible"}) 
    all_perms = "" 
    d.setdefault(one_username + " (" + name + ")", []) 
    for value in find_all: 
     test = value.find("a", {"onmouseover": True}) 
     if test is not None: 
      if "MyAppID" in test.text: 
       d[one_username + " (" + name + ")"].append(test.text) 

    for group in groups: 
     try: 
      d[one_username + " (" + name + ")"].append(check_output("net user /domain " + one_username + "| find \"" + group + "\"", shell=True, stderr=subprocess.STDOUT).strip().decode("utf-8")) 
     except Exception: 
      pass

答

（这个答案会忽略HTML解析你的代码做...你可以队列成池相同，以这种方式如何排队net user调用）

首先，让我们定义一个函数，它的一个(user, group)和tuple返回所需的信息。现在

# a function that calls net user to find info on a (user, group) 
def get_group_info(usr_grp): 
    # unpack the arguments 
    usr, grp = usr_grp 

    try: 
     return (usr, grp, 
       check_output(
        "net user /domain " + usr + "| find \"" + grp + "\"", 
        shell=True, 
        stderr=subprocess.STDOUT 
        ).strip().decode("utf-8"))) 
    except Exception: 
     return (usr, grp, None)

，我们可以使用multiprocessing.dummy.Pool

from multiprocessing.dummy import Pool 
import itertools 

# create a pool with four worker threads 
pool = Pool(4) 

# run get_group_info for every user, group 
async_result = pool.map_async(get_group_info, itertools.product(user_list, groups)) 

# now do some other work we care about 
... 

# and then wait on our results 
results = async_result.get()

的results是(user, group, data)元组列表中的一个线程池中运行这一点，你的愿望都可以加工。

注：此代码是目前未测试由于在平台

答

看起来好像是producer consumer problem。

主线程应生成任务

class Task: 
    def Task(self,user,group) 
     self.user = user 
     self.group = group 
    def run(self): 
     pass # call command with self.user and self.group and process results 

twp = TaskWorkerPool(4) 
for group in groups: 
    twp.add(Task(user,group)) 
twp.wait()

答

在Python 3的差，更简单的和方便的解决方案是使用concurrent.futures。

concurrent.futures模块为异步执行可调用对象提供了一个高级接口。 Reference...

import concurrent.futures 


# Get a list containing all groups of a user 
def get_groups(username): 
    # Do the request and check here 
    # And return the groups of current user with a list 
    return list() 

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: 
    # Mark each future with its groups 
    future_to_groups = {executor.submit(get_groups, user): user 
         for user in user_list} 

    # Now it comes to the result of each user 
    for future in concurrent.futures.as_completed(future_to_groups): 
     user = future_to_groups[future] 
     try: 
      # Receive the returned result of current user 
      groups = future.result() 
     except Exception as exc: 
      print('%r generated an exception: %s' % (user, exc)) 
     else: 
      # Here you do anything you need on `groups` 
      # Output or collect them 
      print('%r is in %d groups' % (user, len(groups)))

注意max_workers这里指线程的最大数量。

见here哪里这个例子从何而来。

编辑：

如果你需要做的每个检查在单独的线程：

import concurrent.futures 


# Check if a `user` is in a `group` 
def check(user, group): 
    # Do the check here 
    # And return True if user is in this group, False if not 
    return True 

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: 
    # Mark each future with its user and group 
    future_to_checks = {executor.submit(check, user, group): (user, group) 
         for user in user_list for group in group_list} 

    # Now it comes to the result of each check 
    # The try-except-else clause is omitted here 
    for future in concurrent.futures.as_completed(future_to_checks): 
     user, group = future_to_checks[future] 
     in_group = future.result() 
     if in_group is True: 
      print('%r is in %r' % (user, group))

通过@donkopotamus启发，itertools.product这里可以用来生成所有目标。

如果你不需要处理异常，它会更简单：

import concurrent.futures 
from itertools import product 
from collections import defaultdict 


def check(target): 
    user, group = target 
    return True 

with concurrent.futures.ThreadPoolExecutor() as executor: 
    results = defaultdict(list) 
    targets = list(product(user_list, group_list)) 
    for (user, group), in_group in zip(targets, executor.map(check, targets)): 
     if in_group is True: 
      results[user].append(group) 

    print(results)

如何进行未来调用并等到Python完成？

相关推荐