threading vs multiprocessing vs asyncio -- when each one actually helps
Three different concurrency tools solving three different problems, and the GIL is the reason they're not interchangeable: threading doesn't speed up CPU-bound work (only one thread runs Python bytecode at a time, ever). multiprocessing sidesteps the GIL entirely by using separate processes, each with its own interpreter and GIL -- real parallelism, at the cost of process-startup overhead and no shared memory by default. asyncio is single-threaded cooperative concurrency, built for I/O-bound work where you're mostly waiting, not computing.
import time, threading, multiprocessing
def cpu_bound(n):
total = 0
for i in range(n):
total += i * i
return total
N = 15_000_000
start = time.perf_counter()
cpu_bound(N); cpu_bound(N)
print(f"sequential (2x cpu_bound): {time.perf_counter()-start:.2f}s")
start = time.perf_counter()
t1 = threading.Thread(target=cpu_bound, args=(N,))
t2 = threading.Thread(target=cpu_bound, args=(N,))
t1.start(); t2.start(); t1.join(); t2.join()
print(f"2 threads: {time.perf_counter()-start:.2f}s <- essentially the SAME, the GIL serializes them")
start = time.perf_counter()
p1 = multiprocessing.Process(target=cpu_bound, args=(N,))
p2 = multiprocessing.Process(target=cpu_bound, args=(N,))
p1.start(); p2.start(); p1.join(); p2.join()
print(f"2 processes: {time.perf_counter()-start:.2f}s <- real parallelism, ~half the time")
But for I/O-bound work -- where the CPU is idle, just waiting -- asyncio lets many waits overlap on a single thread, no processes or real parallelism needed:
import asyncio
async def io_task(n):
await asyncio.sleep(0.3) # simulates a network/disk wait -- yields control while waiting
return n
async def main():
start = time.perf_counter()
await asyncio.gather(*(io_task(i) for i in range(10)))
print(f"10 concurrent 0.3s waits via asyncio.gather: {time.perf_counter()-start:.2f}s")
await main()
start = time.perf_counter()
for i in range(10):
time.sleep(0.3)
print(f"10 SEQUENTIAL 0.3s waits: {time.perf_counter()-start:.2f}s")
Rule of thumb: CPU-bound (crunching numbers, parsing, compression) → multiprocessing. I/O-bound with many concurrent waits (web requests, DB queries) → asyncio. threading's niche is mostly I/O-bound code that must interoperate with blocking, non-async libraries.