Calling Python Script With Subprocess.popen And Flushing The Data

Ok so i've seen dozen of threads like that , but none of them gives a complete answer and everything i tried so far foes not work for me. 1) Script that constantly outputs some dat

Solution 1:

As @Warren Weckesser's comment says, your problem is unrelated to buffering issues.

.readline() in the parent process won't return until it reads a newline or reaches EOF. Your child process doesn't print any newlines at all so your parent process doesn't print anything until the child process ends.

The minimal fix is just to remove comma at the end of print i, in the child script.

This also works:

#!/usr/bin/env pythonimport sys
from subprocess import Popen, PIPE

p = Popen([sys.executable or'python',
           '-u', # unbuffer stdout (or make it line-buffered on Python 3)'-c',
           """
import time

for i in range(5):
    print(i) # <-- no comma i.e., each number is on its own line
    time.sleep(1)
"""], stdout=PIPE, bufsize=1)
for line initer(p.stdout.readline, b''):
    print(int(line)**2)

Example:

$ python parent.py
 014916

The numbers are printed every seconds without waiting for the child process to end.

If you don't want to change the child script then you should use readline() that stops at whitespace instead of a newline character e.g.:

#!/usr/bin/env pythonimport sys
from subprocess import Popen, PIPE

p = Popen(['python2', 'child.py'], stdout=PIPE, bufsize=0)
for token in generate_tokens(p.stdout):
    print(int(token))

where generate_tokens() yields whitespace-separated tokens:

defgenerate_tokens(pipe):
    buf = []
    whileTrue:
        b = pipe.read(1) # read one byteifnot b: # EOF
            pipe.close()
            if buf:
                yieldb''.join(buf)
            returnelifnot b.isspace(): # grow token
            buf.append(b)
        elif buf: # full token readyieldb''.join(buf)
            buf = []

It also prints integers as soon as they are printed by the child.