Pythonsubprocess：cmd退出时的callback

我正在启动一个程序使用subprocess.Popen(cmd, shell=TRUE)

我对Python相当陌生，但是觉得应该有一些API让我做类似的事情：

 subprocess.Popen(cmd, shell=TRUE, postexec_fn=function_to_call_on_exit) 

我正在这样做，以便function_to_call_on_exit可以基于知道cmd已经退出（例如保持当前正在运行的外部进程数）

我假设我可以相当简单地将子Popen.wait()封装在一个将线程与Popen.wait()方法结合在一起的类中，但由于我还没有在Python中进行线程化，所以似乎这可能足以使API存在，我想我会试着find一个。

提前致谢 ：）

Answer 1

你是对的 – 这里没有好的API。 你的第二点也是正确的 – devise一个可以为你使用线程的函数是非常容易的。

 import threading import subprocess def popenAndCall(onExit, popenArgs): """ Runs the given args in a subprocess.Popen, and then calls the function onExit when the subprocess completes. onExit is a callable object, and popenArgs is a list/tuple of args that would give to subprocess.Popen. """ def runInThread(onExit, popenArgs): proc = subprocess.Popen(*popenArgs) proc.wait() onExit() return thread = threading.Thread(target=runInThread, args=(onExit, popenArgs)) thread.start() # returns immediately after the thread starts return thread 

即使线程在Python中也是非常容易的，但是请注意，如果onExit（）在计算上花费很大，那么你需要把它放在一个单独的进程中，而不是使用多处理（这样GIL不会减慢你的程序速度）。 实际上它非常简单 – 基本上你可以把所有的调用都replace为threading.Thread和multiprocessing.Process 。 threading.Thread ，因为它们跟随（几乎）相同的API。

Answer 2

我修改了Daniel G的答案，只是简单地将subprocess.Popen args和kwargs作为自己传递，而不是作为单独的tupple / list，因为我想在subprocess.Popen中使用关键字参数。

在我的情况下，我有一个方法postExec() ，我想运行subprocess.Popen('exe', cwd=WORKING_DIR)

用下面的代码，它只是变成popenAndCall(postExec, 'exe', cwd=WORKING_DIR)

 import threading import subprocess def popenAndCall(onExit, *popenArgs, **popenKWArgs): """ Runs a subprocess.Popen, and then calls the function onExit when the subprocess completes. Use it exactly the way you'd normally use subprocess.Popen, except include a callable to execute as the first argument. onExit is a callable object, and *popenArgs and **popenKWArgs are simply passed up to subprocess.Popen. """ def runInThread(onExit, popenArgs, popenKWArgs): proc = subprocess.Popen(*popenArgs, **popenKWArgs) proc.wait() onExit() return thread = threading.Thread(target=runInThread, args=(onExit, popenArgs, popenKWArgs)) thread.start() return thread # returns immediately after the thread starts 

Answer 3

在Python 3.2中有concurrent.futures模块（可以通过pip install futures为旧Python提供<3.2）：

 pool = Pool(max_workers=1) f = pool.submit(subprocess.call, "sleep 2; echo done", shell=True) f.add_done_callback(callback) 

该callback将在与f.add_done_callback()相同的进程中调用。

完整的程序

 import logging import subprocess # to install run `pip install futures` on Python <3.2 from concurrent.futures import ThreadPoolExecutor as Pool info = logging.getLogger(__name__).info def callback(future): if future.exception() is not None: info("got exception: %s" % future.exception()) else: info("process returned %d" % future.result()) def main(): logging.basicConfig( level=logging.INFO, format=("%(relativeCreated)04d %(process)05d %(threadName)-10s " "%(levelname)-5s %(msg)s")) # wait for the process completion asynchronously info("begin waiting") pool = Pool(max_workers=1) f = pool.submit(subprocess.call, "sleep 2; echo done", shell=True) f.add_done_callback(callback) pool.shutdown(wait=False) # no .submit() calls after that point info("continue waiting asynchronously") if __name__=="__main__": main() 

产量

 $ python . && python3 . 0013 05382 MainThread INFO begin waiting 0021 05382 MainThread INFO continue waiting asynchronously done 2025 05382 Thread-1 INFO process returned 0 0007 05402 MainThread INFO begin waiting 0014 05402 MainThread INFO continue waiting asynchronously done 2018 05402 Thread-1 INFO process returned 0 

Answer 4

我有同样的问题，并解决了它使用multiprocessing.Pool 。 有两个诡计：

1. 使池1的大小
2. 在一个长度为1的迭代中传递可迭代的参数

结果是一个完成callback的函数

 def sub(arg): print arg #prints [1,2,3,4,5] return "hello" def cb(arg): print arg # prints "hello" pool = multiprocessing.Pool(1) rval = pool.map_async(sub,([[1,2,3,4,5]]),callback =cb) (do stuff) pool.close() 

就我而言，我希望调用也是非阻塞的。 精美的作品

Answer 5

我受到Daniel G.的启发，并且实现了一个非常简单的用例 – 在我的工作中，我经常需要用不同的参数对同一个（外部）进程进行重复调用。 我已经破解了一个方法来确定每个特定的调用何时完成，但现在我有一个更简洁的方式来发出callback。

我喜欢这个实现，因为它非常简单，但它允许我发出对多个处理器的asynchronous调用（注意我使用multiprocessing而不是threading ）并在完成时接收通知。

我testing了示例程序，效果很好。 请随意编辑并提供反馈。

 import multiprocessing import subprocess class Process(object): """This class spawns a subprocess asynchronously and calls a `callback` upon completion; it is not meant to be instantiated directly (derived classes are called instead)""" def __call__(self, *args): # store the arguments for later retrieval self.args = args # define the target function to be called by # `multiprocessing.Process` def target(): cmd = [self.command] + [str(arg) for arg in self.args] process = subprocess.Popen(cmd) # the `multiprocessing.Process` process will wait until # the call to the `subprocess.Popen` object is completed process.wait() # upon completion, call `callback` return self.callback() mp_process = multiprocessing.Process(target=target) # this call issues the call to `target`, but returns immediately mp_process.start() return mp_process if __name__ == "__main__": def squeal(who): """this serves as the callback function; its argument is the instance of a subclass of Process making the call""" print "finished %s calling %s with arguments %s" % ( who.__class__.__name__, who.command, who.args) class Sleeper(Process): """Sample implementation of an asynchronous process - define the command name (available in the system path) and a callback function (previously defined)""" command = "./sleeper" callback = squeal # create an instance to Sleeper - this is the Process object that # can be called repeatedly in an asynchronous manner sleeper_run = Sleeper() # spawn three sleeper runs with different arguments sleeper_run(5) sleeper_run(2) sleeper_run(1) # the user should see the following message immediately (even # though the Sleeper calls are not done yet) print "program continued" 

示例输出：

 program continued finished Sleeper calling ./sleeper with arguments (1,) finished Sleeper calling ./sleeper with arguments (2,) finished Sleeper calling ./sleeper with arguments (5,) 

下面是sleeper.c的源代码 – 我的样例“耗时”的外部过程

 #include<stdlib.h> #include<unistd.h> int main(int argc, char *argv[]){ unsigned int t = atoi(argv[1]); sleep(t); return EXIT_SUCCESS; } 

编译为：

 gcc -o sleeper sleeper.c 

Answer 6

AFAIK没有这样的API，至less不在subprocess模块中。 你需要自己推出一些东西，可能使用线程。