Multiprocessing Basics

The multiprocessing module creates real OS processes, each with its own Python interpreter and memory. That gives true parallelism for CPU-bound work and crash isolation for risky code. The API mirrors threading: a Process class with target, start, and join, plus the __name__ == "__main__" guard you always need on Windows and macOS.

A Basic Process

import multiprocessing
import os

def worker(name, value):
    print(f"worker {name} in pid {os.getpid()} -> {value * 2}")

if __name__ == "__main__":
    procs = []
    for i in range(3):
        p = multiprocessing.Process(target=worker, args=(f"W{i}", i))
        p.start()
        procs.append(p)

    for p in procs:
        p.join()
    print("main pid:", os.getpid(), "- all workers done")

Each worker prints a different PID. The if __name__ == "__main__" guard is not optional style: on spawn platforms (Windows, macOS with the default start method) the child re-imports your module, and without the guard it would start the processes again, recursively.

Start Methods

ctx = multiprocessing.get_context("fork")    # Linux default, fastest
# "spawn" - new interpreter, child imports the module (Windows/macOS default)
# "forkserver" - spawn from a clean server process

fork copies the parent's memory (fast, but inherits locks and open files), spawn starts clean (slow, but predictable), and forkserver is a compromise. Whatever the method, functions and arguments passed to the worker must be picklable, which means they must be defined at module top level - closures and lambdas cannot cross the process boundary.

Daemon Processes

Process(daemon=True) marks the child for automatic termination when the parent exits. Use daemons for helper workers (heartbeats, log flushers) that should not outlive the application. Non-daemon children keep the parent's join() meaningful; a daemon that is killed mid-write can leave partial data, so make daemon work idempotent.

Exit Codes

p = multiprocessing.Process(target=crashy_function)
p.start()
p.join()
print(p.exitcode)   # 0 for clean exit, non-zero for an unhandled exception

A child that dies from an unhandled exception sets a non-zero exitcode but prints its traceback to the child's stderr - the parent must check the code if the failure matters.

Summary

Use multiprocessing.Process for CPU-bound parallelism, guard the entry point with __main__, keep target functions top-level and picklable, and check exitcode after join() to catch worker crashes.