多线程队列操作不比单线程速度更快
问题描述:
我在python中设计了一个多线程例程,其中一个函数将一个函数添加到一个队列中,并且多个帮助程序线程concurrentlt将事情从队列中弹出,直到它为空。从理论上讲,这应该比单线程实现快得多,但是在我为这个问题设计的实际应用和玩具例子中,情况并非如此。我猜测原因是与Queue对象(根据python文档是线程安全的)有一些同步问题,但这只是一个猜测。任何有关一般优化的帮助表示赞赏!多线程队列操作不比单线程速度更快
还有玩具例子代码:
from Queue import Queue
import time
queue = Queue(maxsize=0)
counter = []
#called by an external main method, adds and removes from global queue in single thread
def single_thread():
fillit()
time_origin = "%.10f" % time.time()
while not queue.empty():
queue.get()
time_end = "%.10f" % time.time()
time_t = float(time_end) - float(time_origin)
print "time of single threaded implementation: " + str(time_t) +"\n"
#called by an external main method; adds to queue and removes from queue in multiple threads
def multi_thread():
fillit()
time_origin = "%.10f" % time.time()
spawn_threads(4)
time_end = "%.10f" % time.time()
time_t = float(time_end) - float(time_origin)
print "time of multi threaded implementation: " + str(time_t) +"\n"
#Fills up the queue with 2^19 elements
def fillit():
for i in range(2 ** 19):
queue.put(i)
#Spawns n helper threads to help empty the queue
def spawn_threads(num_threads):
for i in range(num_threads):
counter.append(0)
thread = myThread(i, "Thread-" + str(i))
thread.setDaemon(True)
thread.start()
while not queue.empty():
continue
print "done with threads " + str(counter) +" elements removed!"
#THREADING SUPPORT CODE
import threading
class myThread (threading.Thread):
def __init__(self, threadID, name):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
def run(self): #each thread continues to empty the queue while it still has elements
while not queue.empty():
queue.get()
global counter
counter[self.threadID] +=1
结果如下:
time of single threaded implementation: 1.51300001144
done with threads [131077, 131070, 131071, 131070] elements removed!
time of multi threaded implementation: 7.77100014687
为什么要将开始和结束时间转换为字符串,只是将它们转换回浮点数以计算已用时间? –
@BrendanAbel你说什么在开始时间内是有效的。这当然与我正在经历的放缓无关。 – jaykk128