ProcessPoolExecutor

concurrent.futures.ProcessPoolExecutor is to processes what ThreadPoolExecutor is to threads: a managed pool of worker processes with map/submit/as_completed. For CPU-bound fan-out it is usually the best default - you get parallelism, result futures, and clean shutdown without managing Process objects by hand.

map Over a CPU-Bound Function

import time
from concurrent.futures import ProcessPoolExecutor

def is_prime(n):
    if n < 2:
        return False
    i = 2
    while i * i <= n:
        if n % i == 0:
            return False
        i += 1
    return True

nums = [10**6 + i for i in range(8)]

t0 = time.perf_counter()
with ProcessPoolExecutor(max_workers=4) as pool:
    results = list(pool.map(is_prime, nums))
print(results, f"in {time.perf_counter() - t0:.2f}s on 4 cores")

Each chunk runs in a separate interpreter, so the GIL is not a bottleneck and the wall-clock time approaches serial time divided by the number of workers (minus pickling overhead).

submit and Futures

For heterogeneous work, submit gives you a future per call. Note the top-level function requirement: the callable and every argument are pickled to the worker, so closures, lambdas, and class methods bound to unpicklable state will raise PicklingError:

def heavy(task_id, factor):
    # long CPU computation
    return task_id, factor ** 2

with ProcessPoolExecutor(max_workers=3) as pool:
    futures = [pool.submit(heavy, i, 10 ** i) for i in range(6)]
    for fut in as_completed(futures):
        print(fut.result())

The Lower-Level Pool

multiprocessing.pool.Pool is the older, more explicit API. It adds starmap (arguments as tuples), imap/imap_unordered (lazy result iteration for very large inputs), and apply_async per-task futures:

from multiprocessing import Pool

def square(x):
    return x * x

with Pool(4) as pool:
    print(list(pool.starmap(pow, [(2, i) for i in range(5)])))
    for r in pool.imap_unordered(square, range(100_000)):
        pass  # stream results as they finish

Sizing the Pool

Default to os.cpu_count() for pure Python computation. Drop below the core count when each worker also waits on I/O or when the machine runs other load; going far above it usually hurts because of process memory and scheduling overhead. For very small tasks, pickling overhead can make the pool slower than a loop - measure first.

Summary

Use ProcessPoolExecutor (or Pool when you need imap/lazy streaming) for CPU-bound fan-out. Keep functions and arguments picklable and top-level, size the pool to the cores, and benchmark against a serial loop when tasks are tiny.