Doc/lib/libpopen2.tex

   1 \section{\module{popen2} ---
   2          Subprocesses with accessible I/O streams}
   3
   4 \declaremodule{standard}{popen2}
   5   \platform{Unix, Windows}
   6 \modulesynopsis{Subprocesses with accessible standard I/O streams.}
   7 \sectionauthor{Drew Csillag}{drew_csillag@geocities.com}
   8
   9
  10 This module allows you to spawn processes and connect to their
  11 input/output/error pipes and obtain their return codes under
  12 \UNIX{} and Windows.
  13
  14 The \module{subprocess} module provides more powerful facilities for
  15 spawning new processes and retrieving their results.  Using the
  16 \module{subprocess} module is preferable to using the \module{popen2}
  17 module.
  18
  19 The primary interface offered by this module is a trio of factory
  20 functions.  For each of these, if \var{bufsize} is specified,
  21 it specifies the buffer size for the I/O pipes.  \var{mode}, if
  22 provided, should be the string \code{'b'} or \code{'t'}; on Windows
  23 this is needed to determine whether the file objects should be opened
  24 in binary or text mode.  The default value for \var{mode} is
  25 \code{'t'}.
  26
  27 On \UNIX, \var{cmd} may be a sequence, in which case arguments will be passed
  28 directly to the program without shell intervention (as with
  29 \function{os.spawnv()}). If \var{cmd} is a string it will be passed to the
  30 shell (as with \function{os.system()}).
  31
  32 The only way to retrieve the return codes for the child processes is
  33 by using the \method{poll()} or \method{wait()} methods on the
  34 \class{Popen3} and \class{Popen4} classes; these are only available on
  35 \UNIX.  This information is not available when using the
  36 \function{popen2()}, \function{popen3()}, and \function{popen4()}
  37 functions, or the equivalent functions in the \refmodule{os} module.
  38 (Note that the tuples returned by the \refmodule{os} module's functions
  39 are in a different order from the ones returned by the \module{popen2}
  40 module.)
  41
  42 \begin{funcdesc}{popen2}{cmd\optional{, bufsize\optional{, mode}}}
  43 Executes \var{cmd} as a sub-process.  Returns the file objects
  44 \code{(\var{child_stdout}, \var{child_stdin})}.
  45 \end{funcdesc}
  46
  47 \begin{funcdesc}{popen3}{cmd\optional{, bufsize\optional{, mode}}}
  48 Executes \var{cmd} as a sub-process.  Returns the file objects
  49 \code{(\var{child_stdout}, \var{child_stdin}, \var{child_stderr})}.
  50 \end{funcdesc}
  51
  52 \begin{funcdesc}{popen4}{cmd\optional{, bufsize\optional{, mode}}}
  53 Executes \var{cmd} as a sub-process.  Returns the file objects
  54 \code{(\var{child_stdout_and_stderr}, \var{child_stdin})}.
  55 \versionadded{2.0}
  56 \end{funcdesc}
  57
  58
  59 On \UNIX, a class defining the objects returned by the factory
  60 functions is also available.  These are not used for the Windows
  61 implementation, and are not available on that platform.
  62
  63 \begin{classdesc}{Popen3}{cmd\optional{, capturestderr\optional{, bufsize}}}
  64 This class represents a child process.  Normally, \class{Popen3}
  65 instances are created using the \function{popen2()} and
  66 \function{popen3()} factory functions described above.
  67
  68 If not using one of the helper functions to create \class{Popen3}
  69 objects, the parameter \var{cmd} is the shell command to execute in a
  70 sub-process.  The \var{capturestderr} flag, if true, specifies that
  71 the object should capture standard error output of the child process.
  72 The default is false.  If the \var{bufsize} parameter is specified, it
  73 specifies the size of the I/O buffers to/from the child process.
  74 \end{classdesc}
  75
  76 \begin{classdesc}{Popen4}{cmd\optional{, bufsize}}
  77 Similar to \class{Popen3}, but always captures standard error into the
  78 same file object as standard output.  These are typically created
  79 using \function{popen4()}.
  80 \versionadded{2.0}
  81 \end{classdesc}
  82
  83 \subsection{Popen3 and Popen4 Objects \label{popen3-objects}}
  84
  85 Instances of the \class{Popen3} and \class{Popen4} classes have the
  86 following methods:
  87
  88 \begin{methoddesc}{poll}{}
  89 Returns \code{-1} if child process hasn't completed yet, or its return
  90 code otherwise.
  91 \end{methoddesc}
  92
  93 \begin{methoddesc}{wait}{}
  94 Waits for and returns the status code of the child process.  The
  95 status code encodes both the return code of the process and
  96 information about whether it exited using the \cfunction{exit()}
  97 system call or died due to a signal.  Functions to help interpret the
  98 status code are defined in the \refmodule{os} module; see section
  99 \ref{os-process} for the \function{W\var{*}()} family of functions.
 100 \end{methoddesc}
 101
 102
 103 The following attributes are also available:
 104
 105 \begin{memberdesc}{fromchild}
 106 A file object that provides output from the child process.  For
 107 \class{Popen4} instances, this will provide both the standard output
 108 and standard error streams.
 109 \end{memberdesc}
 110
 111 \begin{memberdesc}{tochild}
 112 A file object that provides input to the child process.
 113 \end{memberdesc}
 114
 115 \begin{memberdesc}{childerr}
 116 A file object that provides error output from the child process, if
 117 \var{capturestderr} was true for the constructor, otherwise
 118 \code{None}.  This will always be \code{None} for \class{Popen4}
 119 instances.
 120 \end{memberdesc}
 121
 122 \begin{memberdesc}{pid}
 123 The process ID of the child process.
 124 \end{memberdesc}
 125
 126
 127 \subsection{Flow Control Issues \label{popen2-flow-control}}
 128
 129 Any time you are working with any form of inter-process communication,
 130 control flow needs to be carefully thought out.  This remains the case
 131 with the file objects provided by this module (or the \refmodule{os}
 132 module equivalents).
 133
 134 % Example explanation and suggested work-arounds substantially stolen
 135 % from Martin von Löwis:
 136 % http://mail.python.org/pipermail/python-dev/2000-September/009460.html
 137
 138 When reading output from a child process that writes a lot of data to
 139 standard error while the parent is reading from the child's standard
 140 output, a deadlock can occur.  A similar situation can occur with other
 141 combinations of reads and writes.  The essential factors are that more
 142 than \constant{_PC_PIPE_BUF} bytes are being written by one process in
 143 a blocking fashion, while the other process is reading from the other
 144 process, also in a blocking fashion.
 145
 146 There are several ways to deal with this situation.
 147
 148 The simplest application change, in many cases, will be to follow this
 149 model in the parent process:
 150
 151 \begin{verbatim}
 152 import popen2
 153
 154 r, w, e = popen2.popen3('python slave.py')
 155 e.readlines()
 156 r.readlines()
 157 r.close()
 158 e.close()
 159 w.close()
 160 \end{verbatim}
 161
 162 with code like this in the child:
 163
 164 \begin{verbatim}
 165 import os
 166 import sys
 167
 168 # note that each of these print statements
 169 # writes a single long string
 170
 171 print >>sys.stderr, 400 * 'this is a test\n'
 172 os.close(sys.stderr.fileno())
 173 print >>sys.stdout, 400 * 'this is another test\n'
 174 \end{verbatim}
 175
 176 In particular, note that \code{sys.stderr} must be closed after
 177 writing all data, or \method{readlines()} won't return.  Also note
 178 that \function{os.close()} must be used, as \code{sys.stderr.close()}
 179 won't close \code{stderr} (otherwise assigning to \code{sys.stderr}
 180 will silently close it, so no further errors can be printed).
 181
 182 Applications which need to support a more general approach should
 183 integrate I/O over pipes with their \function{select()} loops, or use
 184 separate threads to read each of the individual files provided by
 185 whichever \function{popen*()} function or \class{Popen*} class was
 186 used.
 187
 188 \begin{seealso}
 189   \seemodule{subprocess}{Module for spawning and managing subprocesses.}
 190 \end{seealso}