如何在Python中分配工作给一堆工作线程？

为了将工作分配给一堆工作线程，使用concurrent.futures模块，特别是ThreadPoolExecutor类。

如果你想要对调度算法进行精细的控制，可以手动编写自己的逻辑。使用队列模块创建包含作业列表的队列。队列类维护一个对象列表，具有.put(obj)方法将项添加到队列中，具有.get()方法将它们返回。该类将处理必要的锁定，以确保每个作业仅分配一次。

示例

下面是一个例子 –

import threading, queue, time

# 工作线程从队列中获取作业。当队列为空时，它
# 假定没有更多的工作，并退出。
def worker():
   print('正在运行的工人')
   time.sleep(0.1)
   while True:
      try:
         arg = q.get(block=False)
      except queue.Empty:
         print('工人', threading.current_thread(), end=' ')
         print('队列为空')
         break
      else:
         print('工人', threading.current_thread(), end=' ')
         print('运行的参数', arg)
         time.sleep(0.5)

# 创建一个队列
q = queue.Queue()

# 启动一个5个工人的线程池
for i in range(5):
   t = threading.Thread(target=worker, name='工人 %i' % (i+1))
   t.start()

# 开始向队列添加工作
for i in range(50):
   q.put(i)

# 让线程有时间运行
print('主线程睡眠中')
time.sleep(5)

Running worker
Running worker
Running worker
Running worker
Running worker
Main thread sleeping
Worker  running with argument 0
Worker  running with argument 1
Worker  running with argument 2
Worker  running with argument 3
Worker  running with argument 4
Worker  running with argument 5
Worker  running with argument 6
Worker  running with argument 7
Worker  running with argument 8
Worker  running with argument 9
Worker  running with argument 10
Worker  running with argument 11
Worker  running with argument 12
Worker  running with argument 13
Worker  running with argument 14
Worker  running with argument 15
Worker  running with argument 16
Worker  running with argument 17
Worker  running with argument 18
Worker  running with argument 19
Worker  running with argument 20
Worker  running with argument 21
Worker  running with argument 22
Worker  running with argument 23
Worker  running with argument 24
Worker  running with argument 25
Worker  running with argument 26
Worker  running with argument 28
Worker  running with argument 29
Worker  running with argument 27
Worker  running with argument 30
Worker  running with argument 31
Worker  running with argument 32
Worker  running with argument 33
Worker  running with argument 34
Worker  running with argument 35
Worker  running with argument 36
Worker  running with argument 37
Worker  running with argument 38
Worker  running with argument 39
Worker  running with argument 40
Worker  running with argument 41
Worker  running with argument 42
Worker  running with argument 43
Worker  running with argument 44
Worker  running with argument 45
Worker  running with argument 46
Worker  running with argument 47
Worker  running with argument 48
Worker  running with argument 49
Worker  queue empty
Worker  queue empty
Worker  queue empty
Worker  queue empty
Worker  queue empty