赶上Ctrl + C / SIGINT并在python中优雅退出多进程

如何在多进程python程序中捕获Ctrl + C并正常退出所有进程,我需要解决scheme在unix和windows上都能运行。 我已经尝试了以下内容:

import multiprocessing import time import signal import sys jobs = [] def worker(): signal.signal(signal.SIGINT, signal_handler) while(True): time.sleep(1.1234) print "Working..." def signal_handler(signal, frame): print 'You pressed Ctrl+C!' # for p in jobs: # p.terminate() sys.exit(0) if __name__ == "__main__": for i in range(50): p = multiprocessing.Process(target=worker) jobs.append(p) p.start() 

这是一种工作,但我不认为这是正确的解决scheme。

编辑:这可能是这个的重复

以前接受的解决scheme有竞争条件,它不适用于mapasyncfunction。

处理Ctrl + C / SIGINTmultiprocessing.Pool的正确方法是:

  1. 在创build进程Pool之前,让进程忽略SIGINT 。 这种方式创build的subprocessinheritance了SIGINT处理程序。
  2. 在创buildPool之后,还原父进程中的原始SIGINT处理程序。
  3. 使用map_asyncapply_async来代替阻塞mapapply
  4. 等待超时的结果,因为默认阻止等待忽略所有信号。 这是Python的bug https://bugs.python.org/issue8296

把它放在一起:

 #!/bin/env python from __future__ import print_function import multiprocessing import os import signal import time def run_worker(delay): print("In a worker process", os.getpid()) time.sleep(delay) def main(): print("Initializng 2 workers") original_sigint_handler = signal.signal(signal.SIGINT, signal.SIG_IGN) pool = multiprocessing.Pool(2) signal.signal(signal.SIGINT, original_sigint_handler) try: print("Starting 2 jobs of 5 seconds each") res = pool.map_async(run_worker, [5, 5]) print("Waiting for results") res.get(60) # Without the timeout this blocking call ignores all signals. except KeyboardInterrupt: print("Caught KeyboardInterrupt, terminating workers") pool.terminate() else: print("Normal termination") pool.close() pool.join() if __name__ == "__main__": main() 

正如@YakovShklarov所指出的那样,在忽略信号并在父进程中忽略信号的时间之间存在一段时间,在此期间信号可能会丢失。 使用pthread_sigmask代替临时阻止父进程中的信号传递可以防止信号丢失,但是它在Python-2中不可用。

解决scheme是基于这个链接和这个链接 ,它解决了这个问题,我不得不移动到Pool

 import multiprocessing import time import signal import sys def init_worker(): signal.signal(signal.SIGINT, signal.SIG_IGN) def worker(): while(True): time.sleep(1.1234) print "Working..." if __name__ == "__main__": pool = multiprocessing.Pool(50, init_worker) try: for i in range(50): pool.apply_async(worker) time.sleep(10) pool.close() pool.join() except KeyboardInterrupt: print "Caught KeyboardInterrupt, terminating workers" pool.terminate() pool.join() 

只需在您的工作进程中处理KeyboardInterrupt-SystemExitexception:

 def worker(): while(True): try: msg = self.msg_queue.get() except (KeyboardInterrupt, SystemExit): print("Exiting...") break