Tailing a real-time log file

sh has the ability to respond to subprocesses in an event-driven fashion. A typical example of where this would be useful is tailing a log file for a specific pattern, then responding to that value immediately:

from sh import tail

for line in tail("-f", "info.log", _iter=True):
    if "ERROR" in line:

The _iter special kwarg takes a command that would normally block until completion, and turns its output into a real-time iterable.

Of course, you can do more than just tail log files. Any program that produces output can be iterated over. Say you wanted to send an email to a coworker if their C code emits a warning:

from sh import gcc, git

for line in gcc("-o", "awesome_binary", "awesome_source.c", _iter=True):
    if "warning" in line:
        # parse out the relevant info
        filename, line, char, message = line.split(":", 3)

        # find the commit using git
        commit = git("blame", "-e", filename, L="%d,%d" % (line,line))

        # send them an email
        email_address = parse_email_from_commit_line(commit)
        send_email(email_address, message)

Using _iter is a great way to respond to events from another program, but your blocks while you’re looping, making you unable to do anything else. To be truly event-driven, sh provides callbacks:

from sh import tail

def process_log_line(line):
    if "ERROR" in line:

process = tail("-f", "info.log", _out=process_log_line, _bg=True)

# ... do other stuff here ...


The _out special kwarg lets you to assign a callback to STDOUT. This callback will receive each line of output from tail -f and allow you to do the same processing that we did earlier.

See also

Output Callbacks

See also