显示子过程输出到标准输出并redirect
我正在通过Python的subprocess模块运行一个脚本。 目前我使用：
p = subprocess.Popen('/path/to/script', stdout=subprocess.PIPE, stderr=subprocess.PIPE) result = p.communicate() 然后我将结果打印到标准输出。 这一切都很好，但由于脚本需要很长时间才能完成，所以我希望从脚本实时输出到stdout。 我输出输出的原因是因为我想parsing它。
将subprocess的stdout保存到一个variables中进行进一步处理， 并在subprocess运行时显示它 ：
#!/usr/bin/env python3 from io import StringIO from subprocess import Popen, PIPE with Popen('/path/to/script', stdout=PIPE, bufsize=1, universal_newlines=True) as p, StringIO() as buf: for line in p.stdout: print(line, end='') buf.write(line) output = buf.getvalue() rc = p.returncode
要保存这两个subprocess的stdout和stderr更为复杂，因为您应该同时使用两个stream以避免死锁 ：
stdout_buf, stderr_buf = StringIO(), StringIO() rc = teed_call('/path/to/script', stdout=stdout_buf, stderr=stderr_buf, universal_newlines=True) output = stdout_buf.getvalue() ...
这里定义了teed_call() 。
更新：这是一个更简单的asyncio版本 。
旧版：
下面是一个基于tulip child_process.py示例的单线程解决scheme：
import asyncio import sys from asyncio.subprocess import PIPE @asyncio.coroutine def read_and_display(*cmd): """Read cmd's stdout, stderr while displaying them as they arrive.""" # start process process = yield from asyncio.create_subprocess_exec(*cmd, stdout=PIPE, stderr=PIPE) # read child's stdout/stderr concurrently stdout, stderr = [], [] # stderr, stdout buffers tasks = { asyncio.Task(process.stdout.readline()): ( stdout, process.stdout, sys.stdout.buffer), asyncio.Task(process.stderr.readline()): ( stderr, process.stderr, sys.stderr.buffer)} while tasks: done, pending = yield from asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED) assert done for future in done: buf, stream, display = tasks.pop(future) line = future.result() if line: # not EOF buf.append(line) # save for later display.write(line) # display in terminal # schedule to read the next line tasks[asyncio.Task(stream.readline())] = buf, stream, display # wait for the process to exit rc = yield from process.wait() return rc, b''.join(stdout), b''.join(stderr)
该脚本运行'/path/to/script命令，并行读取stdout和stderr。 这些行相应地打印到父级的stdout / stderr，并保存为字节串，以供将来处理。 要运行read_and_display()协程，我们需要一个事件循环：
import os if os.name == 'nt': loop = asyncio.ProactorEventLoop() # for subprocess' pipes on Windows asyncio.set_event_loop(loop) else: loop = asyncio.get_event_loop() try: rc, *output = loop.run_until_complete(read_and_display("/path/to/script")) if rc: sys.exit("child failed with '{}' exit code".format(rc)) finally: loop.close()
p.communicate()等待子p.communicate()完成 ，然后一次返回其整个输出。
你有没有尝试过这样的事情，而不是逐行读取subprocess输出？
p = subprocess.Popen('/path/to/script', stdout=subprocess.PIPE, stderr=subprocess.PIPE) for line in p.stdout: # do something with this individual line print line
Popen.communicate doc明确指出：
Note: The data read is buffered in memory, so do not use this method if the data size is large or unlimited.
https://docs.python.org/2/library/subprocess.html#subprocess.Popen.communicate
所以，如果你需要实时输出，你需要使用这样的东西：
stream_p = subprocess.Popen('/path/to/script', stdout=subprocess.PIPE, stderr=subprocess.PIPE) while stream_line in stream_p: #Parse it the way you want print stream_line
