Special Kwargs

These arguments alter a command’s behavior. They are not passed to the program. You can use them on any command that you run, but some may not be used together. sh will tell you if there are conflicts.

To set default special keyword arguments on every command run, you may use Default Arguments.

Controlling Output

_out

Default value: None

What to redirect STDOUT to. If this is a string, it will be treated as a file name. You may also pass a file object (or file-like object), an int (representing a file descriptor, like the result of os.pipe()), a io.StringIO object, or a callable.

import sh
sh.ls(_out="/tmp/output")

See also

Redirection

_err

Default value: None

What to redirect STDERR to. See _out.

_err_to_out

Default value: False

If True, duplicate the file descriptor bound to the process’s STDOUT also to STDERR, effectively causing STDERR and STDOUT to go to the same place.

_encoding

Default value: sh.DEFAULT_ENCODING

The character encoding of the process’s STDOUT. By default, this is the locale’s default encoding.

_decode_errors

New in version 1.07.0.

Default value: "strict"

This is how Python should handle decoding errors of the process’s output. By default, this is "strict", but you can use any value that’s valid to bytes.decode(), such as "ignore".

_tee

New in version 1.07.0.

Default value: None

As of 1.07.0, any time redirection is used, either for STDOUT or STDERR, the respective internal buffers are not filled. For example, if you’re downloading a file and using a callback on STDOUT, the internal STDOUT buffer, nor the pipe buffer be filled with data from STDOUT. This option forces those buffers to be filled anyways, in effect “tee-ing” the output into two places (the callback/redirect handler, and the internal buffers).

Execution

_fg

New in version 1.12.0.

Default value: False

Runs a command in the foreground, meaning it is spawned using os.spawnle(). The current process’s STDIN/OUT/ERR is os.dup2()‘d to the new process and so the new process becomes the foreground of the shell executing the script. This option is basically a shortcut for:

import sh
import sys
sh.your_command(_in=sys.stdin, _out=sys.stdout, _err=sys.stderr)

_bg

Default value: False

Runs a command in the background. The command will return immediately, and you will have to run RunningCommand.wait() on it to ensure it terminates.

_bg_exc

New in version 1.12.9.

Default value: True

Automatically report exceptions for the background command. If you set this to False you should make sure to call RunningCommand.wait() or you may swallow exceptions that happen in the background command.

_env

Default value: None

A dictionary defining the only environment variables that will be made accessible to the process. If not specified, the calling process’s environment variables are used.

Note

This dictionary is the authoritative environment for the process. If you wish to change a single variable in your current environement, you must pass a copy of your current environment with the overriden variable to sh.

See also

Environments

_timeout

Default value: None

How much time, in seconds, we should give the process to complete. If the process does not finish within the timeout, it will be sent the signal defined by _timeout_signal.

_timeout_signal

Default value: signal.SIGKILL

The signal to be sent to the process if _timeout is not None.

_cwd

Default value: None

A string that sets the current working directory of the process.

_ok_code

Default value: 0

Either an integer, a list, or a tuple containing the exit code(s) that are considered “ok”, or in other words: do not raise an exception. Some misbehaved programs use exit codes other than 0 to indicate success.

import sh
sh.weird_program(_ok_code=[0,3,5])

_new_session

Default value: True

Determines if our forked process will be executed in its own session via os.setsid().

Note

If _new_session is False, the forked process will be put into its own group via os.setpgrp(). This way, the forked process, and all of it’s children, are always alone in their own group that may be signalled directly, regardless of the value of _new_session.

_uid

New in version 1.12.0.

Default value: None

The user id to assume before the child process calls os.execv().

_preexec_fn

New in version 1.12.0.

Default value: None

A function to be run directly before the child process calls os.execv(). Typically not used by normal users.

Communication

_in

Default value: None

Specifies an argument for the process to use as its standard input. This may be a string, a queue.Queue, a file-like object, or any iterable.

See also

Input via STDIN

_piped

Default value: None

May be True, "out", or "err". Signals a command that it is being used as the input to another command, so it should return its output incrementally as it receives it, instead of aggregating it all at once.

See also

Advanced Piping

_iter

Default value: None

May be True, "out", or "err". Puts a command in iterable mode. In this mode, you can use a for or while loop to iterate over a command’s output in real-time.

import sh
for line in sh.cat("/tmp/file", _iter=True):
    print(line)

_iter_noblock

Default value: None

Same as _iter, except the loop will not block if there is no output to iterate over. Instead, the output from the command will be errno.EWOULDBLOCK.

import sh
import errno
import time

for line in sh.tail("-f", "stuff.log", _iter_noblock=True):
    if line == errno.EWOULDBLOCK:
        print("doing something else...")
        time.sleep(0.5)
    else:
        print("processing line!")

_with

Default value: False

Explicitly tells us that we’re running a command in a with context. This is only necessary if you’re using a command in a with context and passing parameters to it.

import sh
with sh.contrib.sudo(password="abc123", _with=True):
    print(sh.ls("/root"))

_done

New in version 1.11.0.

Default value: None

A callback that is always called when the command completes, even if it completes with an exit code that would raise an exception. After the callback is run, any exception that would be raised is raised.

The callback is passed the RunningCommand instance, a boolean indicating success, and the exit code.

Here’s an example of using _done to create a multiprocess pool, where sh.your_parallel_command is executed concurrently at no more than 10 at a time:

import sh
from threading import Semaphore

pool = Semaphore(10)

def done(cmd, success, exit_code):
    pool.release()

def do_thing(arg):
    pool.acquire()
    sh.your_parallel_command(arg, _bg=True, _done=done)

procs = []
for arg in range(100):
    procs.append(do_thing(arg))

# essentially a join
[p.wait() for p in procs]

TTYs

_tty_in

Default value: False, meaning a os.pipe() will be used.

If True, sh creates a TTY for STDIN, essentially emulating a terminal, as if your command was entered from the commandline. This is necessary for commands that require STDIN to be a TTY.

_tty_out

Default value: True

If True, sh creates a TTY for STDOUT, otherwise use a os.pipe(). This is necessary for commands that require STDOUT to be a TTY.

_tty_size

Default value: (20, 80)

The (rows, columns) of stdout’s TTY. Changing this may affect how much your program prints per line, for example.

Performance & Optimization

_in_bufsize

Default value: 0

The STDIN buffer size. 0 for unbuffered, 1 for line buffered, anything else for a buffer of that amount.

_out_bufsize

Default value: 1

The STDOUT/ERR buffer size. 0 for unbuffered, 1 for line buffered, anything else for a buffer of that amount.

_internal_bufsize

Default value: 3 * 1024**2 chunks

How much of STDOUT/ERR your command will store internally. This value represents the number of bufsize chunks not the total number of bytes. For example, if this value is 100, and STDOUT is line buffered, you will be able to retrieve 100 lines from STDOUT. If STDOUT is unbuffered, you will be able to retrieve only 100 characters.

_no_out

New in version 1.07.0.

Default value: False

Disables STDOUT being internally stored. This is useful for commands that produce huge amounts of output that you don’t need, that would otherwise be hogging memory if stored internally by sh.

_no_err

New in version 1.07.0.

Default value: False

Disables STDERR being internally stored. This is useful for commands that produce huge amounts of output that you don’t need, that would otherwise be hogging memory if stored internally by sh.

_no_pipe

New in version 1.07.0.

Default value: False

Similar to _no_out, this explicitly tells the sh command that it will never be used for piping its output into another command, so it should not fill its internal pipe buffer with the process’s output. This is also useful for conserving memory.

Program Arguments

These are options that affect how command options are fed into the program.

_long_sep

New in version 1.12.0.

Default value: "="

This is the character(s) that separate a program’s long argument’s key from the value, when using kwargs to specify your program’s long arguments. For example, if your program expects a long argument in the form --name value, the way to achieve this would be to set _long_sep=" ".

import sh
sh.your_program(key=value, _long_sep=" ")

Would send the following list of arguments to your program:

["--key value"]

If your program expects the long argument name to be separate from its value, pass None into _long_sep instead:

import sh
sh.your_program(key=value, _long_sep=None)

Would send the following list of arguments to your program:

["--key", "value"]

_long_prefix

New in version 1.12.0.

Default value: "--"

This is the character(s) that prefix a long argument for the program being run. Some programs use single dashes, for example, and do not understand double dashes.

_arg_preprocess

New in version 1.12.0.

Default value: None

This is an advanced option that allows you to rewrite a command’s arguments on the fly, based on other command arguments, or some other variable. It is really only useful in conjunction with baking, and only currently used when constructing contrib wrappers.

Example:

import sh

def processor(args, kwargs):
    return args, kwargs

my_ls = sh.bake.ls(_arg_preprocess=processor)

Warning

The interface to the _arg_preprocess function may change without warning. It is generally only for internal sh use, so don’t use it unless you absolutely have to.

Misc

_log_msg

Default value: None

New in version 1.12.0.