Monthly Archives: September 2014

Redirect output line by line

I often run into situations where I have a (Perl or Python or Shell) script print a bunch of lines to STDOUT. And this is a one-time throw-away script, so additional effort to configure logging is not really worth it. I will run the script and redirect the output to a log file. Typically something like:

But then the OS will not output the lines to that log file until its buffer is full! So I will need to wait for some time. This is especially painful if my script has a sleep of few seconds in a loop (like when doing bulk writes to a production database, to which our app is also writing live data).

A useful trick I googled and found out was to use stdbuf.

So I run the script with stdbuf and line buffering these days:

Option o is for STDOUT and e is for STDERR. Setting them both to L, we output line by line. If we want to have no buffering at all, we could set them to 0, but this would not be performant!

Voila! Now I can use

and the log file gets populated one line at a time!