Worker process hangs when SystemExit Exception is raised #220
Description
When a worker in a multiprocess.Pool catches SIGTERM and raises SystemExit, the worker logs the signal handling but then hangs indefinitely. If the handler raises a different exception (e.g. ValueError), the process exits immediately as expected.
Observe that the worker logs the SIGTERM catch but the program never exits.
This appears to be catching Exception vs. BaseException (see PR)
hang_test.py
import logging
import os
import signal
from multiprocess import get_context
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s %(levelname)-8s %(message)s",
)
def kill_worker_and_exit(x):
def _handler(signum, frame):
logging.debug(f"Worker {os.getpid()} caught SIGTERM on input={x}")
raise SystemExit("worker received SIGTERM")
signal.signal(signal.SIGTERM, _handler)
os.kill(os.getpid(), signal.SIGTERM)
def multiprocess_hang():
ctx = get_context()
pool = ctx.Pool(processes=2, maxtasksperchild=1)
for _ in pool.imap(kill_worker_and_exit, range(3)):
pass
if __name__ == "__main__":
multiprocess_hang()