2 :mod:`subprocess` --- Subprocess management
3 ===========================================
6 :synopsis: Subprocess management.
7 .. moduleauthor:: Peter Åstrand <astrand@lysator.liu.se>
8 .. sectionauthor:: Peter Åstrand <astrand@lysator.liu.se>
13 The :mod:`subprocess` module allows you to spawn new processes, connect to their
14 input/output/error pipes, and obtain their return codes. This module intends to
15 replace several other, older modules and functions, such as::
23 Information about how the :mod:`subprocess` module can be used to replace these
24 modules and functions can be found in the following sections.
28 :pep:`324` -- PEP proposing the subprocess module
31 Using the subprocess Module
32 ---------------------------
34 This module defines one class called :class:`Popen`:
37 .. class:: Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)
41 *args* should be a string, or a sequence of program arguments. The program
42 to execute is normally the first item in the args sequence or the string if
43 a string is given, but can be explicitly set by using the *executable*
44 argument. When *executable* is given, the first item in the args sequence
45 is still treated by most programs as the command name, which can then be
46 different from the actual executable name. On Unix, it becomes the display
47 name for the executing program in utilities such as :program:`ps`.
49 On Unix, with *shell=False* (default): In this case, the Popen class uses
50 :meth:`os.execvp` to execute the child program. *args* should normally be a
51 sequence. A string will be treated as a sequence with the string as the only
52 item (the program to execute).
54 On Unix, with *shell=True*: If args is a string, it specifies the command string
55 to execute through the shell. If *args* is a sequence, the first item specifies
56 the command string, and any additional items will be treated as additional shell
59 On Windows: the :class:`Popen` class uses CreateProcess() to execute the child
60 program, which operates on strings. If *args* is a sequence, it will be
61 converted to a string using the :meth:`list2cmdline` method. Please note that
62 not all MS Windows applications interpret the command line the same way:
63 :meth:`list2cmdline` is designed for applications using the same rules as the MS
66 *bufsize*, if given, has the same meaning as the corresponding argument to the
67 built-in open() function: :const:`0` means unbuffered, :const:`1` means line
68 buffered, any other positive value means use a buffer of (approximately) that
69 size. A negative *bufsize* means to use the system default, which usually means
70 fully buffered. The default value for *bufsize* is :const:`0` (unbuffered).
72 The *executable* argument specifies the program to execute. It is very seldom
73 needed: Usually, the program to execute is defined by the *args* argument. If
74 ``shell=True``, the *executable* argument specifies which shell to use. On Unix,
75 the default shell is :file:`/bin/sh`. On Windows, the default shell is
76 specified by the :envvar:`COMSPEC` environment variable.
78 *stdin*, *stdout* and *stderr* specify the executed programs' standard input,
79 standard output and standard error file handles, respectively. Valid values
80 are :data:`PIPE`, an existing file descriptor (a positive integer), an
81 existing file object, and ``None``. :data:`PIPE` indicates that a new pipe
82 to the child should be created. With ``None``, no redirection will occur;
83 the child's file handles will be inherited from the parent. Additionally,
84 *stderr* can be :data:`STDOUT`, which indicates that the stderr data from the
85 applications should be captured into the same file handle as for stdout.
87 If *preexec_fn* is set to a callable object, this object will be called in the
88 child process just before the child is executed. (Unix only)
90 If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and
91 :const:`2` will be closed before the child process is executed. (Unix only).
92 Or, on Windows, if *close_fds* is true then no handles will be inherited by the
93 child process. Note that on Windows, you cannot set *close_fds* to true and
94 also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.
96 If *shell* is :const:`True`, the specified command will be executed through the
99 If *cwd* is not ``None``, the child's current directory will be changed to *cwd*
100 before it is executed. Note that this directory is not considered when
101 searching the executable, so you can't specify the program's path relative to
104 If *env* is not ``None``, it must be a mapping that defines the environment
105 variables for the new process; these are used instead of inheriting the current
106 process' environment, which is the default behavior.
110 If specified, *env* must provide any variables required
111 for the program to execute. On Windows, in order to run a
112 `side-by-side assembly`_ the specified *env* **must** include a valid
113 :envvar:`SystemRoot`.
115 .. _side-by-side assembly: http://en.wikipedia.org/wiki/Side-by-Side_Assembly
117 If *universal_newlines* is :const:`True`, the file objects stdout and stderr are
118 opened as text files, but lines may be terminated by any of ``'\n'``, the Unix
119 end-of-line convention, ``'\r'``, the old Macintosh convention or ``'\r\n'``, the
120 Windows convention. All of these external representations are seen as ``'\n'``
121 by the Python program.
125 This feature is only available if Python is built with universal newline support
126 (the default). Also, the newlines attribute of the file objects :attr:`stdout`,
127 :attr:`stdin` and :attr:`stderr` are not updated by the communicate() method.
129 The *startupinfo* and *creationflags*, if given, will be passed to the
130 underlying CreateProcess() function. They can specify things such as appearance
131 of the main window and priority for the new process. (Windows only)
136 Special value that can be used as the *stdin*, *stdout* or *stderr* argument
137 to :class:`Popen` and indicates that a pipe to the standard stream should be
143 Special value that can be used as the *stderr* argument to :class:`Popen` and
144 indicates that standard error should go into the same handle as standard
148 Convenience Functions
149 ^^^^^^^^^^^^^^^^^^^^^
151 This module also defines two shortcut functions:
154 .. function:: call(*popenargs, **kwargs)
156 Run command with arguments. Wait for command to complete, then return the
157 :attr:`returncode` attribute.
159 The arguments are the same as for the Popen constructor. Example::
161 retcode = call(["ls", "-l"])
165 Like :meth:`Popen.wait`, this will deadlock if the child process
166 generates enough output to a stdout or stderr pipe such that it blocks
167 waiting for the OS pipe buffer to accept more data.
170 .. function:: check_call(*popenargs, **kwargs)
172 Run command with arguments. Wait for command to complete. If the exit code was
173 zero then return, otherwise raise :exc:`CalledProcessError`. The
174 :exc:`CalledProcessError` object will have the return code in the
175 :attr:`returncode` attribute.
177 The arguments are the same as for the Popen constructor. Example::
179 check_call(["ls", "-l"])
181 .. versionadded:: 2.5
185 See the warning for :func:`call`.
188 .. function:: check_output(*popenargs, **kwargs)
190 Run command with arguments and return its output as a byte string.
192 If the exit code was non-zero it raises a :exc:`CalledProcessError`. The
193 :exc:`CalledProcessError` object will have the return code in the
195 attribute and output in the :attr:`output` attribute.
197 The arguments are the same as for the :class:`Popen` constructor. Example::
199 >>> subprocess.check_output(["ls", "-l", "/dev/null"])
200 'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
202 The stdout argument is not allowed as it is used internally.
203 To capture standard error in the result, use ``stderr=subprocess.STDOUT``::
205 >>> subprocess.check_output(
206 ["/bin/sh", "-c", "ls non_existent_file ; exit 0"],
207 stderr=subprocess.STDOUT)
208 'ls: non_existent_file: No such file or directory\n'
210 .. versionadded:: 2.7
216 Exceptions raised in the child process, before the new program has started to
217 execute, will be re-raised in the parent. Additionally, the exception object
218 will have one extra attribute called :attr:`child_traceback`, which is a string
219 containing traceback information from the childs point of view.
221 The most common exception raised is :exc:`OSError`. This occurs, for example,
222 when trying to execute a non-existent file. Applications should prepare for
223 :exc:`OSError` exceptions.
225 A :exc:`ValueError` will be raised if :class:`Popen` is called with invalid
228 check_call() will raise :exc:`CalledProcessError`, if the called process returns
229 a non-zero return code.
235 Unlike some other popen functions, this implementation will never call /bin/sh
236 implicitly. This means that all characters, including shell metacharacters, can
237 safely be passed to child processes.
243 Instances of the :class:`Popen` class have the following methods:
246 .. method:: Popen.poll()
248 Check if child process has terminated. Set and return :attr:`returncode`
252 .. method:: Popen.wait()
254 Wait for child process to terminate. Set and return :attr:`returncode`
259 This will deadlock if the child process generates enough output to a
260 stdout or stderr pipe such that it blocks waiting for the OS pipe buffer
261 to accept more data. Use :meth:`communicate` to avoid that.
264 .. method:: Popen.communicate(input=None)
266 Interact with process: Send data to stdin. Read data from stdout and stderr,
267 until end-of-file is reached. Wait for process to terminate. The optional
268 *input* argument should be a string to be sent to the child process, or
269 ``None``, if no data should be sent to the child.
271 :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``.
273 Note that if you want to send data to the process's stdin, you need to create
274 the Popen object with ``stdin=PIPE``. Similarly, to get anything other than
275 ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or
280 The data read is buffered in memory, so do not use this method if the data
281 size is large or unlimited.
284 .. method:: Popen.send_signal(signal)
286 Sends the signal *signal* to the child.
290 On Windows only SIGTERM is supported so far. It's an alias for
293 .. versionadded:: 2.6
296 .. method:: Popen.terminate()
298 Stop the child. On Posix OSs the method sends SIGTERM to the
299 child. On Windows the Win32 API function :cfunc:`TerminateProcess` is called
302 .. versionadded:: 2.6
305 .. method:: Popen.kill()
307 Kills the child. On Posix OSs the function sends SIGKILL to the child.
308 On Windows :meth:`kill` is an alias for :meth:`terminate`.
310 .. versionadded:: 2.6
313 The following attributes are also available:
317 Use :meth:`communicate` rather than :attr:`.stdin.write <stdin>`,
318 :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` to avoid
319 deadlocks due to any of the other OS pipe buffers filling up and blocking the
323 .. attribute:: Popen.stdin
325 If the *stdin* argument was :data:`PIPE`, this attribute is a file object
326 that provides input to the child process. Otherwise, it is ``None``.
329 .. attribute:: Popen.stdout
331 If the *stdout* argument was :data:`PIPE`, this attribute is a file object
332 that provides output from the child process. Otherwise, it is ``None``.
335 .. attribute:: Popen.stderr
337 If the *stderr* argument was :data:`PIPE`, this attribute is a file object
338 that provides error output from the child process. Otherwise, it is
342 .. attribute:: Popen.pid
344 The process ID of the child process.
347 .. attribute:: Popen.returncode
349 The child return code, set by :meth:`poll` and :meth:`wait` (and indirectly
350 by :meth:`communicate`). A ``None`` value indicates that the process
351 hasn't terminated yet.
353 A negative value ``-N`` indicates that the child was terminated by signal
357 .. _subprocess-replacements:
359 Replacing Older Functions with the subprocess Module
360 ----------------------------------------------------
362 In this section, "a ==> b" means that b can be used as a replacement for a.
366 All functions in this section fail (more or less) silently if the executed
367 program cannot be found; this module raises an :exc:`OSError` exception.
369 In the following examples, we assume that the subprocess module is imported with
370 "from subprocess import \*".
373 Replacing /bin/sh shell backquote
374 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
380 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
383 Replacing shell pipeline
384 ^^^^^^^^^^^^^^^^^^^^^^^^
388 output=`dmesg | grep hda`
390 p1 = Popen(["dmesg"], stdout=PIPE)
391 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
392 output = p2.communicate()[0]
395 Replacing :func:`os.system`
396 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
400 sts = os.system("mycmd" + " myarg")
402 p = Popen("mycmd" + " myarg", shell=True)
403 sts = os.waitpid(p.pid, 0)
407 * Calling the program through the shell is usually not required.
409 * It's easier to look at the :attr:`returncode` attribute than the exit status.
411 A more realistic example would look like this::
414 retcode = call("mycmd" + " myarg", shell=True)
416 print >>sys.stderr, "Child was terminated by signal", -retcode
418 print >>sys.stderr, "Child returned", retcode
420 print >>sys.stderr, "Execution failed:", e
423 Replacing the :func:`os.spawn <os.spawnl>` family
424 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
428 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
430 pid = Popen(["/bin/mycmd", "myarg"]).pid
434 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
436 retcode = call(["/bin/mycmd", "myarg"])
440 os.spawnvp(os.P_NOWAIT, path, args)
442 Popen([path] + args[1:])
444 Environment example::
446 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
448 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
451 Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3`
452 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
456 pipe = os.popen(cmd, 'r', bufsize)
458 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout
462 pipe = os.popen(cmd, 'w', bufsize)
464 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
468 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
470 p = Popen(cmd, shell=True, bufsize=bufsize,
471 stdin=PIPE, stdout=PIPE, close_fds=True)
472 (child_stdin, child_stdout) = (p.stdin, p.stdout)
478 child_stderr) = os.popen3(cmd, mode, bufsize)
480 p = Popen(cmd, shell=True, bufsize=bufsize,
481 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
484 child_stderr) = (p.stdin, p.stdout, p.stderr)
488 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
490 p = Popen(cmd, shell=True, bufsize=bufsize,
491 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
492 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
494 Return code handling translates as follows::
496 pipe = os.popen(cmd, 'w')
499 if rc != None and rc % 256:
500 print "There were some errors"
502 process = Popen(cmd, 'w', stdin=PIPE)
504 process.stdin.close()
505 if process.wait() != 0:
506 print "There were some errors"
509 Replacing functions from the :mod:`popen2` module
510 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
514 If the cmd argument to popen2 functions is a string, the command is executed
515 through /bin/sh. If it is a list, the command is directly executed.
519 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
521 p = Popen(["somestring"], shell=True, bufsize=bufsize,
522 stdin=PIPE, stdout=PIPE, close_fds=True)
523 (child_stdout, child_stdin) = (p.stdout, p.stdin)
527 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
529 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
530 stdin=PIPE, stdout=PIPE, close_fds=True)
531 (child_stdout, child_stdin) = (p.stdout, p.stdin)
533 :class:`popen2.Popen3` and :class:`popen2.Popen4` basically work as
534 :class:`subprocess.Popen`, except that:
536 * :class:`Popen` raises an exception if the execution fails.
538 * the *capturestderr* argument is replaced with the *stderr* argument.
540 * ``stdin=PIPE`` and ``stdout=PIPE`` must be specified.
542 * popen2 closes all file descriptors by default, but you have to specify
543 ``close_fds=True`` with :class:`Popen`.