ThreadPoolExecutor

Creating and joining a handful of threads by hand gets noisy fast. The concurrent.futures.ThreadPoolExecutor manages a pool of worker threads for you: you submit callables, the pool runs them on idle workers, and you collect results through Future objects. It is the standard way to run many I/O tasks concurrently in a few lines.

map for Homogeneous Work

from concurrent.futures import ThreadPoolExecutor
import time

def fetch_page(url):
    # simulate a network call
    time.sleep(0.5)
    return f"content of {url}"

urls = [f"https://example.com/page/{i}" for i in range(10)]

with ThreadPoolExecutor(max_workers=5) as pool:
    results = list(pool.map(fetch_page, urls))

print(len(results))  # 10 results, ~2s total instead of ~5s

map preserves input order in the output and streams results as they become available. The pool size should match the resource you are hammering - 5 workers for an API limited to 5 concurrent connections, more for local disk I/O.

submit for Heterogeneous Work

When each task is different, use submit, which returns a Future immediately:

from concurrent.futures import ThreadPoolExecutor, as_completed

def load(key, delay):
    time.sleep(delay)
    return key, delay * 10

with ThreadPoolExecutor(max_workers=4) as pool:
    futures = {pool.submit(load, k, d): k for k, d in
               [("fast", 0.1), ("slow", 0.5), ("medium", 0.3)]}

    for fut in as_completed(futures):
        key, value = fut.result()
        print(key, value)   # prints in completion order, not input order

as_completed yields futures as they finish, which is what you want when you should process results eagerly (update a UI, flush to a database) rather than wait for the slowest task. Call fut.result() to retrieve the value or re-raise the exception the task raised.

shutdown

The context manager calls shutdown(wait=True) automatically. Without it: shutdown(wait=False) lets already-submitted work run while refusing new submissions, and cancel_futures=True (Python 3.9+) also cancels queued-but-not-started tasks. Threads in the pool are joined on shutdown, so a with block is the idiomatic scope for a pool.

Summary

Use ThreadPoolExecutor instead of manual thread bookkeeping: map for a batch of identical calls, submit plus as_completed when you need per-task futures and eager processing, and size the pool to the bottleneck, not to the number of cores.