current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.
up vote 1 down vote favorite

Suppose I have a executable which for which I want to log STDOUT and STDERR to separate files, like so:

python write.py > out 2> err

When I use this command with GNU timeout, my out file is always empty. Why does this happen, and how can I fix this?

timeout 5s python write.py > out 2> err

Example write.py:

#!/bin/bash
import sys, time

i = 0
while True:    
    print i
    print >> sys.stderr, i
    time.sleep(1)
    i += 1
share|improve this question

1 Answer 1

up vote 3 down vote accepted 
$ timeout 5s python write.py > out 2> err
$ ls -l out err
-rw-r--r-- 1 yeti yeti 10 Apr  6 08:12 err
-rw-r--r-- 1 yeti yeti  0 Apr  6 08:12 out
$ timeout 5s python -u write.py > out 2> err
$ ls -l out err
-rw-r--r-- 1 yeti yeti 10 Apr  6 08:13 err
-rw-r--r-- 1 yeti yeti 10 Apr  6 08:13 out
$ cmp err out && echo same
same
$ cat out 
0
1
2
3
4

python uses buffered writes on stdout but not on stderr. So everything written to sys.stderr is written immediately but stuff for sys.stdout is kept in a buffer until the buffer is full to minimise write operations. Within 5 seconds, the buffer does not fill enough to get written even once and timeout terminates the python interpreter.

Adding the -u option to python's invocation, writes to stdout will be unbuffered too.

You can force the same behaviour for a Python program by setting the environment variable PYTHONUNBUFFERED. For other programs you can use unbuffer from the expect package or stdbuf from GNU coreutils (see Turn off buffering in pipe).

share|improve this answer
    
So this problem actually came up in the context of a different executable (not Python), do you know a more general solution? if it helps, the executable was a make compilation using Scala's sbt. –  dal102 Apr 6 at 6:59
2  
@dal102 unbuffer or stdbuf (see my edit) –  Gilles Apr 6 at 22:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.